-- Logs begin at Tue 2022-08-30 14:00:21 UTC, end at Tue 2022-08-30 15:32:13 UTC. -- Aug 30 14:28:13.881159 dv-cinder-828806-5-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 30 14:28:13.890114 dv-cinder-828806-5-iscsi devstack@keystone.service[81679]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 30 14:28:13.892169 dv-cinder-828806-5-iscsi devstack@keystone.service[81679]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Aug 30 14:28:13 2022] *** Aug 30 14:28:13.892169 dv-cinder-828806-5-iscsi devstack@keystone.service[81679]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 30 14:28:13.892169 dv-cinder-828806-5-iscsi devstack@keystone.service[81679]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 30 14:28:13.892169 dv-cinder-828806-5-iscsi devstack@keystone.service[81679]: nodename: dv-cinder-828806-5-iscsi Aug 30 14:28:13.892169 dv-cinder-828806-5-iscsi devstack@keystone.service[81679]: machine: x86_64 Aug 30 14:28:13.892169 dv-cinder-828806-5-iscsi devstack@keystone.service[81679]: clock source: unix Aug 30 14:28:13.892169 dv-cinder-828806-5-iscsi devstack@keystone.service[81679]: pcre jit disabled Aug 30 14:28:13.892169 dv-cinder-828806-5-iscsi devstack@keystone.service[81679]: detected number of CPU cores: 4 Aug 30 14:28:13.892169 dv-cinder-828806-5-iscsi devstack@keystone.service[81679]: current working directory: / Aug 30 14:28:13.892169 dv-cinder-828806-5-iscsi devstack@keystone.service[81679]: detected binary path: /usr/bin/uwsgi-core Aug 30 14:28:13.892693 dv-cinder-828806-5-iscsi devstack@keystone.service[81679]: your processes number limit is 47680 Aug 30 14:28:13.892693 dv-cinder-828806-5-iscsi devstack@keystone.service[81679]: your memory page size is 4096 bytes Aug 30 14:28:13.892693 dv-cinder-828806-5-iscsi devstack@keystone.service[81679]: detected max file descriptor number: 2048 Aug 30 14:28:13.892693 dv-cinder-828806-5-iscsi devstack@keystone.service[81679]: lock engine: pthread robust mutexes Aug 30 14:28:13.892693 dv-cinder-828806-5-iscsi devstack@keystone.service[81679]: thunder lock: enabled Aug 30 14:28:13.892693 dv-cinder-828806-5-iscsi devstack@keystone.service[81679]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 30 14:28:13.892693 dv-cinder-828806-5-iscsi devstack@keystone.service[81679]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 30 14:28:13.933138 dv-cinder-828806-5-iscsi devstack@keystone.service[81679]: Python main interpreter initialized at 0x55e50a62c9c0 Aug 30 14:28:13.933138 dv-cinder-828806-5-iscsi devstack@keystone.service[81679]: python threads support enabled Aug 30 14:28:13.933138 dv-cinder-828806-5-iscsi devstack@keystone.service[81679]: your server socket listen backlog is limited to 100 connections Aug 30 14:28:13.933138 dv-cinder-828806-5-iscsi devstack@keystone.service[81679]: your mercy for graceful operations on workers is 90 seconds Aug 30 14:28:13.933522 dv-cinder-828806-5-iscsi devstack@keystone.service[81679]: mapped 403077 bytes (393 KB) for 2 cores Aug 30 14:28:13.933589 dv-cinder-828806-5-iscsi devstack@keystone.service[81679]: *** Operational MODE: preforking *** Aug 30 14:28:13.933881 dv-cinder-828806-5-iscsi devstack@keystone.service[81679]: *** uWSGI is running in multiple interpreter mode *** Aug 30 14:28:13.933881 dv-cinder-828806-5-iscsi devstack@keystone.service[81679]: spawned uWSGI master process (pid: 81679) Aug 30 14:28:13.934098 dv-cinder-828806-5-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 30 14:28:13.934731 dv-cinder-828806-5-iscsi devstack@keystone.service[81679]: spawned uWSGI worker 1 (pid: 81689, cores: 1) Aug 30 14:28:13.935088 dv-cinder-828806-5-iscsi devstack@keystone.service[81679]: spawned uWSGI worker 2 (pid: 81690, cores: 1) Aug 30 14:28:13.938537 dv-cinder-828806-5-iscsi devstack@keystone.service[81679]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 30 14:28:18.245171 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 30 14:28:18.245171 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: from cryptography.utils import int_from_bytes Aug 30 14:28:18.245878 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 30 14:28:18.245878 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: from cryptography.utils import int_from_bytes Aug 30 14:28:18.246292 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 30 14:28:18.246292 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: from cryptography.utils import int_from_bytes Aug 30 14:28:18.246400 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 30 14:28:18.246400 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: from cryptography.utils import int_from_bytes Aug 30 14:28:18.296156 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 30 14:28:18.296350 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 30 14:28:18.296526 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] command line args: [] {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 30 14:28:18.296786 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 30 14:28:18.296872 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 30 14:28:18.296976 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ================================================================================ {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 30 14:28:18.297063 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 30 14:28:18.297167 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] command line args: [] {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 30 14:28:18.297242 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] admin_token = **** {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.297438 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 30 14:28:18.297515 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] config_dir = [] {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.297617 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ================================================================================ {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 30 14:28:18.297701 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.297817 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] config_source = [] {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.297890 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] admin_token = **** {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.298004 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] debug = True {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.298107 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] config_dir = [] {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.298194 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.298455 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.298455 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] config_source = [] {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.298563 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.298618 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] debug = True {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.298758 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.298841 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.298971 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.299050 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.299298 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.299423 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.299536 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] list_limit = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.299631 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.299729 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] log_config_append = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.299822 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.299919 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.300019 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] list_limit = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.300113 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] log_dir = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.300205 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] log_config_append = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.300303 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] log_file = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.300394 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.300460 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.300582 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] log_dir = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.300582 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] log_file = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.300763 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.300884 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.300959 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.301077 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.301161 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.301285 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.301359 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.301471 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.301552 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.301659 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.301743 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.301846 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.301921 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.302090 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.302177 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.302333 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.302457 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.302566 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.302641 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.302799 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.302929 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.303012 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.303081 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.303278 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.303403 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.303532 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.303614 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.303766 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.304043 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.304160 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.30/identity {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.304242 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] publish_errors = False {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.304444 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.304519 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.30/identity {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.304675 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.304794 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] publish_errors = False {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.304879 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.304955 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.305091 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.305177 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.305295 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.305390 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.305550 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] standard_threads = False {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.305626 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.305786 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.305862 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.306035 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.306111 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] standard_threads = False {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.306214 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.306290 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.306394 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] use_journal = False {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.306502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.306644 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] use_json = False {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.306719 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.306795 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] use_stderr = False {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.306872 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] use_journal = False {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.306947 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] use_syslog = False {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.307018 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] use_json = False {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.307126 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.307224 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] use_stderr = False {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.307353 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] use_syslog = False {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.307428 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.307584 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 14:28:18.307669 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.308090 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.308176 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.308330 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.308411 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.308590 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.308671 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.308829 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.308913 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.309084 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.309164 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.309318 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.309399 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.309566 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.309643 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.309801 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.309882 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.310046 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.310122 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.310306 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.310388 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.310561 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.310730 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.310843 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.311042 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.311138 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.311313 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.311404 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.311555 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.311637 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.311815 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.311900 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.312064 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.312145 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.312304 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.312381 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.312540 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.312693 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.312797 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.312873 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.313041 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.313116 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.313280 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.313357 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.313533 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.313612 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.313766 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.313840 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.314000 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.314083 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.314235 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.314311 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.314473 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.314548 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.314701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.314776 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.314930 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.315005 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.315164 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.315279 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.315427 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.315504 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.315671 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.315753 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.315901 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.315982 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.316129 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.316205 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.316374 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.316449 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.316606 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.316679 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.316842 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.316921 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.317079 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.317160 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.317307 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.317388 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.317539 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.317614 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.317770 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.317845 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.318000 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.318075 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.318217 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.318290 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.318444 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.318520 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.318672 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.318748 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.318911 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.318994 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.319142 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.319234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.319401 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.319478 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.319638 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.319715 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.319867 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.319942 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.320101 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.320182 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.320334 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.320439 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.320571 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.320672 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.320798 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.320898 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.321033 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.321137 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.321266 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.321378 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.321496 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.321574 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.321727 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.321866 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.321945 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.322060 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.322175 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.322303 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.322426 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.322567 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.322659 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.322736 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.322880 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.322969 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.323108 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.323199 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.323384 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.323479 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.323618 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.323705 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.323845 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.323930 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.324082 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.324169 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.324313 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.324386 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.324579 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.324695 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.324813 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.324892 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.325051 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.325152 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.325291 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.325396 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.325524 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.325604 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.325759 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.325836 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.326021 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.326116 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.326269 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.326509 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.326593 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.326760 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.326944 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.327038 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.327193 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.327294 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.327467 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.327541 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.327708 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.327803 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.327944 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.328023 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.328179 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.328256 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.328417 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.328507 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.328653 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.328731 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.328891 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.328970 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.329128 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.329223 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.329371 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.329447 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.329610 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.329695 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.329850 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.330008 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.330083 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.330159 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.330310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.330393 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.330540 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.330626 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.330774 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.330854 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.331004 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.331081 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.331256 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.331397 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.331582 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.331703 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.331839 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.331965 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.332090 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.332214 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.332348 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.332473 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.332597 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.332721 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.332843 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.332967 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.333105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.333231 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.333356 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.333489 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] auth.external = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.333627 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.333752 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.333912 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.334051 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.334190 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.334324 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.334456 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.334586 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] auth.password = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.334711 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] auth.external = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.334836 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] auth.token = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.334968 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.335094 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.335249 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.335384 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.335511 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.335636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.335773 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] auth.password = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.335900 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.336025 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] auth.token = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.336156 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.336282 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.336407 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.336540 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.336662 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.336787 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.336912 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] credential.caching = True {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.337045 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.337176 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.337309 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.337435 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.337561 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.337685 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.337809 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.337932 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.338064 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] credential.caching = True {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.338189 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.338314 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.338447 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.338571 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.338701 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.338842 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.338978 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.339059 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.339184 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.339315 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.339390 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.339469 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.339541 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.339612 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.339682 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.339752 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.339752 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.339885 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.339885 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.340061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.340136 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.340206 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.340276 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] federation.caching = True {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.340355 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.340427 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.340497 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.340567 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.340637 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.340707 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.340785 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.340921 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.341000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] federation.caching = True {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.341172 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.341249 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.341411 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.341498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.342186 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.342297 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.342594 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.342678 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.342874 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.343012 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.343117 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.343289 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.343395 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.343549 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.343639 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] identity.caching = True {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.343787 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.343866 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.344041 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.344120 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.344279 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.344432 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.344529 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.344607 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.344765 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] identity.caching = True {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.344842 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.345014 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.345099 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.345255 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.345331 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.345488 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.345566 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.345724 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.345810 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.345970 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.346047 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.346200 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.346277 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.346433 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.346519 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.346683 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.346792 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.346933 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.347048 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.347183 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.347296 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.347470 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.347548 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.347723 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.347801 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.347975 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.348059 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.348222 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.348300 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.348460 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.348536 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.348696 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.348782 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.348938 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.349016 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.349173 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.349257 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.349413 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.349498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.349656 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.349799 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.349882 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.349955 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.350138 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.350223 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.350369 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.350447 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.350604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.350681 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.350845 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.350975 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.351081 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.351157 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.351354 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.351433 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.351610 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.351702 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.351820 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.351899 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.352061 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.352138 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.352293 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.352379 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.352532 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.352609 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.352773 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.352852 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.353002 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.353080 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.353260 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.353366 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.353473 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.353554 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.353710 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.353797 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.353948 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.354026 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.354183 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.354260 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.354418 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.354498 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.354653 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.354731 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.354899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.354976 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.355136 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.355236 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.355389 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.355467 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.355620 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.355709 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.355852 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.user = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.355938 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.356097 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.356176 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.356329 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.356406 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.356564 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.356644 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.356799 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.356878 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.357029 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.user = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.357107 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.357913 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.358005 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.358179 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.358412 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.358492 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.358725 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.358813 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.358984 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.359063 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.359240 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.359321 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.359482 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.359573 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.359733 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.359810 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.359966 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.360044 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.360204 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.360291 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.360441 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.360520 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.360678 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.360755 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.360907 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.360993 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.361154 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.361231 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.361395 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.361473 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.361635 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.361721 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.361876 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.361954 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.362114 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.362193 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.362343 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.362429 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.362589 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.362668 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.362824 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.362909 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.363057 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.363145 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.363315 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.363396 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.363549 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.363628 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.363790 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.363888 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.364031 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.364111 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.364264 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.364342 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.364517 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.364607 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.364747 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.364853 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.364990 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.365092 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.365226 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.365334 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.365458 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.365581 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] resource.caching = True {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.365693 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.365837 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.365937 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.366091 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.366171 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.366332 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.366411 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.366587 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.366667 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.366841 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.366922 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] resource.caching = True {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.367083 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.367162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.367351 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.367439 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.367586 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.367667 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.367832 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.367910 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.368072 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] role.caching = True {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.368152 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.368300 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.368375 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] role.driver = sql {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.368534 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.368613 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.368768 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.368858 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.369006 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.369085 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.369241 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] role.caching = True {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.369320 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.369493 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] role.driver = sql {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.369584 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.369722 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.369801 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.369966 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.370045 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.370211 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.370298 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.370457 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.370535 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.370691 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.370769 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.370932 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.371019 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.371176 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.371278 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.371444 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.371524 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.371689 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.371780 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.371919 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.372093 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.372172 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.372335 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.372413 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.372590 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.372674 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.372877 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.372956 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.373328 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.373532 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.373621 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.373834 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.373914 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.374077 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.374156 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.374336 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.374423 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.374583 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.374661 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.374819 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.374898 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.375061 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.375141 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.375305 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.375399 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.375563 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.375642 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.375815 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.375895 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.376052 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.376131 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.376287 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.376366 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.376522 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.376603 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.376755 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.376832 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.376994 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] token.caching = True {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.377073 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.377235 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.377316 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.377482 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.377559 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.377720 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.377797 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.377949 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.378029 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.378185 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] token.caching = True {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.378264 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.378423 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.378502 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.378671 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.378752 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.378908 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.378987 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.379160 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.379256 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.379441 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.379523 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.379677 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.379754 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.379915 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.379992 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.380155 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.380235 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.380404 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.380483 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.380640 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.380719 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.380877 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.380970 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.381111 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.381191 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.381357 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.381436 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] database.connection = **** {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.381624 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.381710 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.381852 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.381939 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.382094 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.382175 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.382335 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.382413 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.382575 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.382663 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] database.connection = **** {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.382815 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.382894 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.383054 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.383133 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.383316 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.383410 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.383560 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.383640 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.383795 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.383874 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.384031 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.384129 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.384265 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.384344 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.384498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.384577 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.384738 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.384824 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.384972 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.385051 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.385205 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.385284 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.385437 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.385525 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.385674 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.385751 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.385846 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81689) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 30 14:28:18.385925 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.386135 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.386365 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.386603 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.386837 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 14:28:18.387006 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81690) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 30 14:28:18.415370 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81689) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.415604 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81689) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.415831 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81689) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.416221 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81690) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.416453 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81690) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.416675 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81690) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.424840 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81689) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.425964 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81690) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.454478 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81689) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.454791 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81689) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.457251 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81690) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.457529 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81690) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.510661 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81689) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.510825 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81690) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.560515 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81689) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.560748 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81689) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.561053 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81690) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.561297 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81690) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.715651 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81689) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.715905 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81689) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.716112 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81689) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.716339 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81689) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.716616 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81689) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.717349 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81689) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.717804 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81690) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.718210 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81690) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.718449 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81690) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.718673 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81690) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.718891 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81690) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.719104 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81690) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.740220 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81690) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.740448 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81690) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.740531 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81689) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.740669 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81690) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.740752 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81689) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.740887 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81690) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.740970 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81689) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.741101 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81690) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.741192 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81689) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.741312 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81690) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.741394 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81689) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.741525 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81690) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.741607 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81689) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.741786 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81689) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.752910 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81690) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.753010 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81689) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.766395 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81689) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.766505 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81690) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.766583 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81689) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.766702 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81690) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.766786 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81689) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.766927 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81690) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.767011 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81689) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.767146 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81690) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.767259 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81689) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.767389 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81690) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.767471 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81689) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.767611 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81690) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.767701 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81689) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.767814 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81690) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 14:28:18.784468 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.784682 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 30 14:28:18.785030 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.785112 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.785237 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 30 14:28:18.785335 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 30 14:28:18.785522 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.785605 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.785791 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.785873 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 30 14:28:18.786059 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.786143 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.786319 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.786400 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.786831 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.787098 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.796613 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.796898 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.797178 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.797340 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.797470 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.797620 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.797891 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.798175 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.804489 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81689) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:28:18.805089 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81690) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:28:18.807401 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81689) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:28:18.807693 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.808217 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81690) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:28:18.808299 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.808503 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.808584 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.808785 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.808865 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.809053 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.809136 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.809328 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.809417 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.809585 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.809666 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.809854 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.809933 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.810130 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.810212 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.810977 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.811153 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.811672 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.811953 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.833674 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.834496 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.835144 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81689) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:28:18.835458 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.835981 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81690) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:28:18.836267 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.840001 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81689) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:28:18.840290 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.840578 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.840882 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81690) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:28:18.841166 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.841446 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.847841 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81689) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:28:18.848134 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.848541 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81690) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:28:18.848835 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.852104 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81689) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:28:18.852401 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.852693 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.852876 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81690) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:28:18.852985 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.853167 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.853249 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.853453 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.853734 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.854007 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.854307 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.854575 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.854844 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.855244 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.855538 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.855834 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.871891 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.872178 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.872850 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.873348 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.876110 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81689) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:28:18.877391 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81690) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:28:18.879232 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.879536 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.880611 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.880910 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.883888 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81689) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:28:18.885083 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81690) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:28:18.887010 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81689) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:28:18.888209 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81690) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:28:18.890183 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.890487 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.890779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.891069 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.891381 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.891754 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.891899 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.892058 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.892346 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.892642 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.892934 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.893223 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.909840 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81690) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:28:18.910056 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81689) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:28:18.910139 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.910355 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.910435 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.910634 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.910714 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.910921 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.917542 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.917685 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.919059 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.919357 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.921452 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.921657 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.921738 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.921943 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.924363 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81690) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:28:18.924626 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81689) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:28:18.924707 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.924921 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.925049 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.925229 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.925335 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.925521 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.925610 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.925817 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.937403 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81690) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:28:18.937535 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81689) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:28:18.937685 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.937827 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.938064 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.938159 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.938348 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.938431 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.938632 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.938713 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.938933 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.939089 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.939245 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.939329 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.954603 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81689) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:28:18.954836 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81690) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:28:18.957656 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81689) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:28:18.957741 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81690) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:28:18.960573 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.960656 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.962010 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.962092 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.963529 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81689) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:28:18.963610 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81690) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:28:18.963821 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.963901 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.964119 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.964202 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.971326 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.971415 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.972807 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81690) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:28:18.972899 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81689) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:28:18.975700 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.975840 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.976004 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.976085 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.976298 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.976391 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.976594 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.976678 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.985271 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81690) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:28:18.985369 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81689) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:28:18.985561 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.985644 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.985844 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.985925 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.992706 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81690) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:28:18.992848 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81689) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 14:28:18.992992 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.993073 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.993273 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.993354 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.993552 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.993642 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.993825 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.993906 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.994098 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.994178 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.994368 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.994459 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.994647 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.994730 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.994926 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.995009 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.995224 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.995306 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.995495 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.995586 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.995756 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.995837 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.996019 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.996098 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.996301 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81690) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:18.996389 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81689) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 14:28:19.030987 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55e50a62c9c0 pid: 81689 (default app) Aug 30 14:28:19.033883 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55e50a62c9c0 pid: 81690 (default app) Aug 30 14:28:19.034523 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d9220-ae75-418f-9412-9a9d1932d38f None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:19.034859 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d9220-ae75-418f-9412-9a9d1932d38f None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:19.035180 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d9220-ae75-418f-9412-9a9d1932d38f None None] PATH_INFO: `/v3` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:19.039374 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1/1] 10.222.0.30 () {50 vars in 791 bytes} [Tue Aug 30 14:28:19 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:25.623099 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0baff0-2242-482f-b149-f4e84de7b57c None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:25.623635 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0baff0-2242-482f-b149-f4e84de7b57c None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:25.623742 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0baff0-2242-482f-b149-f4e84de7b57c None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:25.627941 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1/2] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:28:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:28:25.633138 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43ddec6-ebe6-4375-96a4-3c684a88164f None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:25.633599 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43ddec6-ebe6-4375-96a4-3c684a88164f None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:25.633739 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43ddec6-ebe6-4375-96a4-3c684a88164f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:25.704919 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: INFO dbcounter [None req-f43ddec6-ebe6-4375-96a4-3c684a88164f None None] Registered counter for database keystone Aug 30 14:28:25.724227 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG oslo_db.sqlalchemy.engines [None req-f43ddec6-ebe6-4375-96a4-3c684a88164f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81689) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 30 14:28:25.730278 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writer thread running {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 30 14:28:25.908341 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG passlib.handlers.bcrypt [None req-f43ddec6-ebe6-4375-96a4-3c684a88164f None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81689) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 30 14:28:25.908658 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG passlib.handlers.bcrypt [None req-f43ddec6-ebe6-4375-96a4-3c684a88164f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81689) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 30 14:28:25.967610 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-f43ddec6-ebe6-4375-96a4-3c684a88164f None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:28:25.969228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-f43ddec6-ebe6-4375-96a4-3c684a88164f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:28:26.097329 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 2/3] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:28:25 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 465 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 30 14:28:26.583762 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0baff0-2242-482f-b149-f4e84de7b57c None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:26.653293 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: INFO dbcounter [None req-1d0baff0-2242-482f-b149-f4e84de7b57c None None] Registered counter for database keystone Aug 30 14:28:26.674347 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG oslo_db.sqlalchemy.engines [None req-1d0baff0-2242-482f-b149-f4e84de7b57c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81690) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 30 14:28:26.680554 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writer thread running {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 30 14:28:26.799496 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0baff0-2242-482f-b149-f4e84de7b57c None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:26.803810 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ce013-ddc7-42ec-9a17-671ed592825e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:26.804431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ce013-ddc7-42ec-9a17-671ed592825e admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:26.804712 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ce013-ddc7-42ec-9a17-671ed592825e admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:26.804985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ce013-ddc7-42ec-9a17-671ed592825e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:26.814073 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ce013-ddc7-42ec-9a17-671ed592825e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:26.864711 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: WARNING keystone.common.rbac_enforcer.enforcer [None req-8c5ce013-ddc7-42ec-9a17-671ed592825e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 30 14:28:26.864987 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ce013-ddc7-42ec-9a17-671ed592825e admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:26.875097 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 2/4] 10.222.0.30 () {58 vars in 1324 bytes} [Tue Aug 30 14:28:26 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 30 14:28:26.880974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43ddec6-ebe6-4375-96a4-3c684a88164f None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:26.895482 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43ddec6-ebe6-4375-96a4-3c684a88164f None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:26.899923 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edcb8d9-602d-4e80-abfb-f39d9340dc38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:26.901067 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edcb8d9-602d-4e80-abfb-f39d9340dc38 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:26.901356 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edcb8d9-602d-4e80-abfb-f39d9340dc38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:26.901632 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edcb8d9-602d-4e80-abfb-f39d9340dc38 admin admin] PATH_INFO: `/v3/projects/2ac908dcbf994029b1c39e157d0de0cb` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:26.903178 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edcb8d9-602d-4e80-abfb-f39d9340dc38 admin admin] RBAC: Authorizing `identity:get_project(project_id=2ac908dcbf994029b1c39e157d0de0cb)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:26.956074 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: WARNING keystone.common.rbac_enforcer.enforcer [None req-0edcb8d9-602d-4e80-abfb-f39d9340dc38 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 30 14:28:26.956418 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edcb8d9-602d-4e80-abfb-f39d9340dc38 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:26.960966 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 3/5] 10.222.0.30 () {56 vars in 1207 bytes} [Tue Aug 30 14:28:26 2022] GET /identity/v3/projects/2ac908dcbf994029b1c39e157d0de0cb => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:28.886728 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69007be3-fed8-4cbf-8d1e-b788edbff9f7 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:28.887361 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69007be3-fed8-4cbf-8d1e-b788edbff9f7 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:28.887361 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69007be3-fed8-4cbf-8d1e-b788edbff9f7 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:28.891509 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 3/6] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:28:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:28:28.896921 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc1e266-54ca-4e77-ba63-b0b3bd4432be None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:28.897329 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc1e266-54ca-4e77-ba63-b0b3bd4432be None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:28.897477 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc1e266-54ca-4e77-ba63-b0b3bd4432be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:28.917689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-0bc1e266-54ca-4e77-ba63-b0b3bd4432be None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:28:28.919236 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-0bc1e266-54ca-4e77-ba63-b0b3bd4432be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:28:28.927907 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 4/7] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:28:28 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 30 14:28:29.195732 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69007be3-fed8-4cbf-8d1e-b788edbff9f7 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:29.208370 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69007be3-fed8-4cbf-8d1e-b788edbff9f7 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:29.212657 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970b2680-8158-4474-86ef-3d56cb9d5eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:29.213255 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b2680-8158-4474-86ef-3d56cb9d5eff admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:29.213536 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b2680-8158-4474-86ef-3d56cb9d5eff admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:29.213810 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b2680-8158-4474-86ef-3d56cb9d5eff admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:29.222306 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970b2680-8158-4474-86ef-3d56cb9d5eff admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:29.224128 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970b2680-8158-4474-86ef-3d56cb9d5eff admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:29.232926 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 4/8] 10.222.0.30 () {58 vars in 1324 bytes} [Tue Aug 30 14:28:29 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 14:28:29.238483 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc1e266-54ca-4e77-ba63-b0b3bd4432be None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:29.247510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc1e266-54ca-4e77-ba63-b0b3bd4432be None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:29.251930 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d88265-17a6-450c-a7ea-6c96e01084e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:29.252545 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d88265-17a6-450c-a7ea-6c96e01084e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:29.252823 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d88265-17a6-450c-a7ea-6c96e01084e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:29.253095 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d88265-17a6-450c-a7ea-6c96e01084e9 admin admin] PATH_INFO: `/v3/users/4a989d2c0a4a49eb9303e4db6cc0b7e8` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:29.254390 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d88265-17a6-450c-a7ea-6c96e01084e9 admin admin] RBAC: Authorizing `identity:get_user(user_id=4a989d2c0a4a49eb9303e4db6cc0b7e8)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:29.256250 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d88265-17a6-450c-a7ea-6c96e01084e9 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:29.257762 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 5/9] 10.222.0.30 () {56 vars in 1198 bytes} [Tue Aug 30 14:28:29 2022] GET /identity/v3/users/4a989d2c0a4a49eb9303e4db6cc0b7e8 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:31.224975 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4ea23-3320-434c-ae48-664e5ef80dba None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:31.225475 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4ea23-3320-434c-ae48-664e5ef80dba None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:31.225581 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4ea23-3320-434c-ae48-664e5ef80dba None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:31.226878 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 5/10] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:28:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:28:31.232616 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb30fe-7c75-43e5-975e-4b52fab853ac None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:31.233181 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb30fe-7c75-43e5-975e-4b52fab853ac None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:31.233181 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb30fe-7c75-43e5-975e-4b52fab853ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:31.251528 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-f3bb30fe-7c75-43e5-975e-4b52fab853ac None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:28:31.252715 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-f3bb30fe-7c75-43e5-975e-4b52fab853ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:28:31.276294 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0924b-ddfd-45b3-b05d-523125d577b3 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:31.276577 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0924b-ddfd-45b3-b05d-523125d577b3 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:31.276850 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0924b-ddfd-45b3-b05d-523125d577b3 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:31.278164 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 6/11] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:28:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:28:31.283345 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fbf83a-d597-444c-9168-69945b7f18f4 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:31.283614 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fbf83a-d597-444c-9168-69945b7f18f4 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:31.283880 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fbf83a-d597-444c-9168-69945b7f18f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:31.326563 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG passlib.handlers.bcrypt [None req-50fbf83a-d597-444c-9168-69945b7f18f4 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81690) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 30 14:28:31.327050 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG passlib.handlers.bcrypt [None req-50fbf83a-d597-444c-9168-69945b7f18f4 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81690) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 30 14:28:31.333644 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 6/12] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:28:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:28:31.376819 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-50fbf83a-d597-444c-9168-69945b7f18f4 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:28:31.378422 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-50fbf83a-d597-444c-9168-69945b7f18f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:28:31.485710 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 7/13] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:28:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:28:31.603676 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bb30fe-7c75-43e5-975e-4b52fab853ac None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:31.627769 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:31.708968 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bb30fe-7c75-43e5-975e-4b52fab853ac None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:31.742228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbc4006-f18d-44e7-81b7-e3213baea9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:31.742801 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc4006-f18d-44e7-81b7-e3213baea9c1 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:31.743082 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc4006-f18d-44e7-81b7-e3213baea9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:31.743386 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc4006-f18d-44e7-81b7-e3213baea9c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:31.748478 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fbf83a-d597-444c-9168-69945b7f18f4 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:31.754363 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbc4006-f18d-44e7-81b7-e3213baea9c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:28:31.763169 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-8dbc4006-f18d-44e7-81b7-e3213baea9c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:28:31.763169 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:28:31.763169 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:28:31.763169 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:28:31.763169 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:28:31.763169 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:28:31.763169 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:28:31.763169 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:28:31.763169 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:28:31.763169 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:28:31.763169 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:28:31.763169 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:28:31.763169 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:28:31.763169 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:28:31.763169 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:28:31.763169 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:28:31.764284 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:28:31.764284 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:28:31.764284 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:28:31.764284 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:28:31.764284 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:28:31.764284 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:28:31.764284 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:28:31.764284 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:28:31.764284 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:28:31.764284 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:28:31.764284 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:28:31.764284 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:28:31.764284 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:28:31.764284 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:28:31.764284 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:28:31.764284 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:28:31.765099 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:28:31.765099 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:28:31.765099 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:28:31.765099 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:28:31.765099 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:28:31.765099 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:28:31.765099 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:28:31.765099 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:28:31.765099 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:28:31.765099 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:28:31.765099 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:28:31.765099 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:28:31.765099 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 7/14] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 30 14:28:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:28:31.770255 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbc4006-f18d-44e7-81b7-e3213baea9c1 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:31.852287 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fbf83a-d597-444c-9168-69945b7f18f4 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:31.853030 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:31.854988 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:31.872837 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbc4006-f18d-44e7-81b7-e3213baea9c1 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:31.890410 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800c39c1-3768-4a82-838f-a5145972a720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:31.891278 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800c39c1-3768-4a82-838f-a5145972a720 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:31.891561 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800c39c1-3768-4a82-838f-a5145972a720 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:31.891828 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800c39c1-3768-4a82-838f-a5145972a720 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:31.907294 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d80971-d027-41c2-b4d5-e67c8f9317da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:31.907906 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d80971-d027-41c2-b4d5-e67c8f9317da None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:31.908185 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d80971-d027-41c2-b4d5-e67c8f9317da None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:31.908463 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d80971-d027-41c2-b4d5-e67c8f9317da None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:31.909130 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d80971-d027-41c2-b4d5-e67c8f9317da None admin] RBAC: Adding query filter params (name=admin) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:28:31.909432 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d80971-d027-41c2-b4d5-e67c8f9317da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:31.910817 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d80971-d027-41c2-b4d5-e67c8f9317da None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:31.923093 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 8/15] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:28:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:28:31.928457 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d80971-d027-41c2-b4d5-e67c8f9317da None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:31.964167 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800c39c1-3768-4a82-838f-a5145972a720 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:31.965690 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800c39c1-3768-4a82-838f-a5145972a720 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:32.028247 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d80971-d027-41c2-b4d5-e67c8f9317da None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:32.062263 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 8/16] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:28:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 14:28:32.062435 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f079ac8-36bc-4d1e-bd2c-a50b4d7c6a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:32.063288 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f079ac8-36bc-4d1e-bd2c-a50b4d7c6a7b None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:32.063572 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f079ac8-36bc-4d1e-bd2c-a50b4d7c6a7b None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:32.063841 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f079ac8-36bc-4d1e-bd2c-a50b4d7c6a7b None admin] PATH_INFO: `/v3/users/4a989d2c0a4a49eb9303e4db6cc0b7e8` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:32.065010 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f079ac8-36bc-4d1e-bd2c-a50b4d7c6a7b None admin] RBAC: Authorizing `identity:get_user(user_id=4a989d2c0a4a49eb9303e4db6cc0b7e8)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:32.066459 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f079ac8-36bc-4d1e-bd2c-a50b4d7c6a7b None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:32.067837 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800c39c1-3768-4a82-838f-a5145972a720 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:32.068065 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 9/17] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:28:31 2022] GET /identity/v3/users/4a989d2c0a4a49eb9303e4db6cc0b7e8 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:28:32.070598 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:32.073437 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f079ac8-36bc-4d1e-bd2c-a50b4d7c6a7b None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:32.088656 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:32.167555 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800c39c1-3768-4a82-838f-a5145972a720 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:32.180181 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f079ac8-36bc-4d1e-bd2c-a50b4d7c6a7b None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:32.202154 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7089668-2aa3-43bb-83f7-7ba97cce8956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:32.202998 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7089668-2aa3-43bb-83f7-7ba97cce8956 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:32.203357 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7089668-2aa3-43bb-83f7-7ba97cce8956 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:32.203630 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7089668-2aa3-43bb-83f7-7ba97cce8956 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:32.218320 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f17a09-c206-4a40-afbe-35f26a946232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:32.219140 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f17a09-c206-4a40-afbe-35f26a946232 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:32.219488 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f17a09-c206-4a40-afbe-35f26a946232 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:32.219763 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f17a09-c206-4a40-afbe-35f26a946232 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:32.220886 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f17a09-c206-4a40-afbe-35f26a946232 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:32.222411 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f17a09-c206-4a40-afbe-35f26a946232 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:32.223261 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7089668-2aa3-43bb-83f7-7ba97cce8956 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:32.224147 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 10/18] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:28:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:28:32.224740 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7089668-2aa3-43bb-83f7-7ba97cce8956 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:32.229760 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f17a09-c206-4a40-afbe-35f26a946232 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:32.242939 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-f7089668-2aa3-43bb-83f7-7ba97cce8956 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:28:32.242939 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:28:32.242939 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:28:32.242939 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:28:32.242939 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:28:32.242939 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:28:32.242939 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:28:32.242939 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:28:32.242939 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:28:32.242939 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:28:32.242939 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:28:32.242939 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:28:32.242939 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:28:32.242939 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:28:32.242939 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:28:32.242939 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:28:32.244205 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:28:32.244205 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:28:32.244205 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:28:32.244205 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:28:32.244205 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:28:32.244205 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:28:32.244205 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:28:32.244205 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:28:32.244205 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:28:32.244205 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:28:32.244205 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:28:32.244205 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:28:32.244205 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:28:32.244205 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:28:32.244205 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:28:32.244205 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:28:32.244205 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:28:32.245026 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:28:32.245026 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:28:32.245026 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:28:32.245026 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 9/19] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:28:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:28:32.249775 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7089668-2aa3-43bb-83f7-7ba97cce8956 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:32.306365 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:32.311760 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:32.332419 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f17a09-c206-4a40-afbe-35f26a946232 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:32.353075 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7089668-2aa3-43bb-83f7-7ba97cce8956 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:32.367317 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830fefb4-d0ec-4388-b911-0762102ef7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:32.367929 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830fefb4-d0ec-4388-b911-0762102ef7e2 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:32.368211 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830fefb4-d0ec-4388-b911-0762102ef7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:32.368487 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830fefb4-d0ec-4388-b911-0762102ef7e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:32.369215 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830fefb4-d0ec-4388-b911-0762102ef7e2 None admin] RBAC: Adding query filter params (role.id=3cda0615d87d4af0860a2a5e9b4e2c05, scope.domain.id=default, user.id=4a989d2c0a4a49eb9303e4db6cc0b7e8) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:28:32.369526 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830fefb4-d0ec-4388-b911-0762102ef7e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:32.370991 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830fefb4-d0ec-4388-b911-0762102ef7e2 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:32.378662 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 11/20] 10.222.0.30 () {56 vars in 1322 bytes} [Tue Aug 30 14:28:32 2022] GET /identity/v3/role_assignments?user.id=4a989d2c0a4a49eb9303e4db6cc0b7e8&scope.domain.id=default&role.id=3cda0615d87d4af0860a2a5e9b4e2c05 => generated 233 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:28:32.386518 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7815f6-4b4d-4462-943a-4b27973de5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:32.387158 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7815f6-4b4d-4462-943a-4b27973de5f2 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:32.387474 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7815f6-4b4d-4462-943a-4b27973de5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:32.387749 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7815f6-4b4d-4462-943a-4b27973de5f2 None admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:32.389154 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7815f6-4b4d-4462-943a-4b27973de5f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:28:32.389654 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7815f6-4b4d-4462-943a-4b27973de5f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:32.391323 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7815f6-4b4d-4462-943a-4b27973de5f2 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:32.412648 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 10/21] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:28:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:28:34.545954 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeeab34-a0f5-4b63-947b-135d1092c60a None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:34.546927 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeeab34-a0f5-4b63-947b-135d1092c60a None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:34.547453 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeeab34-a0f5-4b63-947b-135d1092c60a None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:34.549260 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 12/22] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:28:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:28:34.554763 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99978675-d9ff-4963-8816-0eabed3b878f None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:34.555602 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99978675-d9ff-4963-8816-0eabed3b878f None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:34.556090 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99978675-d9ff-4963-8816-0eabed3b878f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:34.584274 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-99978675-d9ff-4963-8816-0eabed3b878f None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:28:34.585832 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-99978675-d9ff-4963-8816-0eabed3b878f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:28:34.679662 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 11/23] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:28:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:28:34.802649 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0054816e-e9e3-4e47-ad60-4f9d76b0e630 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:34.803351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0054816e-e9e3-4e47-ad60-4f9d76b0e630 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:34.803851 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0054816e-e9e3-4e47-ad60-4f9d76b0e630 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:34.805630 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 13/24] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:28:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:28:34.812130 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930a136-6788-4555-973b-f9b836088074 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:34.812656 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930a136-6788-4555-973b-f9b836088074 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:34.813133 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930a136-6788-4555-973b-f9b836088074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:34.844209 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-f930a136-6788-4555-973b-f9b836088074 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:28:34.845744 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-f930a136-6788-4555-973b-f9b836088074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:28:34.905771 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:34.939743 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 12/25] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:28:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:28:35.021116 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c9e42-782a-4b18-b395-08747023f63f None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:35.021752 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c9e42-782a-4b18-b395-08747023f63f None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:35.022245 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c9e42-782a-4b18-b395-08747023f63f None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:35.024059 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 14/26] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:28:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:28:35.029006 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b219ad0b-0613-442f-82bd-8db2cc01adb0 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:35.029301 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b219ad0b-0613-442f-82bd-8db2cc01adb0 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:35.029580 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b219ad0b-0613-442f-82bd-8db2cc01adb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:35.056196 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-b219ad0b-0613-442f-82bd-8db2cc01adb0 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:28:35.057464 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-b219ad0b-0613-442f-82bd-8db2cc01adb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:28:35.085220 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4053dc6-3438-4859-b52c-8aa041b65b0d None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:35.085503 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4053dc6-3438-4859-b52c-8aa041b65b0d None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:35.085767 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4053dc6-3438-4859-b52c-8aa041b65b0d None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:35.087104 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 15/27] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:28:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:28:35.096638 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5290441-47df-4018-a2f3-e5fddd3858ef None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:35.096923 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5290441-47df-4018-a2f3-e5fddd3858ef None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:35.097204 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5290441-47df-4018-a2f3-e5fddd3858ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:35.131248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-c5290441-47df-4018-a2f3-e5fddd3858ef None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:28:35.131248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-c5290441-47df-4018-a2f3-e5fddd3858ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:28:35.200043 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 13/28] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:28:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:28:35.202065 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b219ad0b-0613-442f-82bd-8db2cc01adb0 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:35.271411 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 16/29] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:28:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 30 14:28:35.306902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5290441-47df-4018-a2f3-e5fddd3858ef None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:35.377345 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b219ad0b-0613-442f-82bd-8db2cc01adb0 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:35.390922 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:35.397850 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:35.425959 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5290441-47df-4018-a2f3-e5fddd3858ef None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:35.468801 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978cc333-aa20-4a13-bb67-23f258207ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:35.469713 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978cc333-aa20-4a13-bb67-23f258207ed6 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:35.470140 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978cc333-aa20-4a13-bb67-23f258207ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:35.470654 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978cc333-aa20-4a13-bb67-23f258207ed6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:35.476906 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbfc66d-7444-4dd5-8de2-a3e3a43760e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:35.477835 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbfc66d-7444-4dd5-8de2-a3e3a43760e7 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:35.478255 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbfc66d-7444-4dd5-8de2-a3e3a43760e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:35.478663 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbfc66d-7444-4dd5-8de2-a3e3a43760e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:35.487288 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978cc333-aa20-4a13-bb67-23f258207ed6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:35.492895 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978cc333-aa20-4a13-bb67-23f258207ed6 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:35.501910 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbfc66d-7444-4dd5-8de2-a3e3a43760e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:28:35.515305 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-9fbfc66d-7444-4dd5-8de2-a3e3a43760e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:28:35.515305 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:28:35.515305 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:28:35.515305 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:28:35.515305 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:28:35.515305 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:28:35.515305 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:28:35.515305 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:28:35.515305 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:28:35.515305 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:28:35.515305 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:28:35.515305 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:28:35.515305 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:28:35.515305 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:28:35.515305 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:28:35.515305 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:28:35.516248 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:28:35.516248 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:28:35.516248 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:28:35.516248 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:28:35.516248 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:28:35.516248 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:28:35.516248 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:28:35.516248 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:28:35.516248 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:28:35.516248 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:28:35.516248 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:28:35.516248 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:28:35.516248 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:28:35.516248 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:28:35.516248 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:28:35.516248 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:28:35.516999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:28:35.516999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:28:35.516999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:28:35.516999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:28:35.516999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:28:35.516999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:28:35.516999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:28:35.516999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:28:35.516999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:28:35.516999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:28:35.516999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:28:35.516999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:28:35.516999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 14/30] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 30 14:28:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:28:35.523230 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbfc66d-7444-4dd5-8de2-a3e3a43760e7 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:35.526063 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-978cc333-aa20-4a13-bb67-23f258207ed6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:28:35.526063 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:28:35.526063 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:28:35.526063 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:28:35.526063 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:28:35.526063 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:28:35.526063 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:28:35.526063 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:28:35.526063 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:28:35.526063 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:28:35.526063 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:28:35.526063 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:28:35.526063 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:28:35.526063 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:28:35.526063 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:28:35.526063 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:28:35.526902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:28:35.526902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:28:35.526902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:28:35.526902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:28:35.526902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:28:35.526902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:28:35.526902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:28:35.526902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:28:35.526902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:28:35.526902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:28:35.526902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:28:35.526902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:28:35.526902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:28:35.526902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:28:35.526902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:28:35.526902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:28:35.526902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:28:35.527826 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:28:35.527826 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:28:35.527826 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:28:35.529607 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 17/31] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:28:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:28:35.535893 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978cc333-aa20-4a13-bb67-23f258207ed6 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:35.709277 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978cc333-aa20-4a13-bb67-23f258207ed6 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:35.712503 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbfc66d-7444-4dd5-8de2-a3e3a43760e7 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:35.746399 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:35.756421 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd835e-5455-41bd-886d-249f2d456907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:35.757250 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd835e-5455-41bd-886d-249f2d456907 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:35.757707 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd835e-5455-41bd-886d-249f2d456907 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:35.758143 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd835e-5455-41bd-886d-249f2d456907 None admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:35.758960 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bd835e-5455-41bd-886d-249f2d456907 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:28:35.759486 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bd835e-5455-41bd-886d-249f2d456907 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:35.761056 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bd835e-5455-41bd-886d-249f2d456907 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:35.761720 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e018b-349b-4cfd-96d6-f81ca6d1de27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:35.762506 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e018b-349b-4cfd-96d6-f81ca6d1de27 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:35.763607 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e018b-349b-4cfd-96d6-f81ca6d1de27 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:35.764091 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e018b-349b-4cfd-96d6-f81ca6d1de27 None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:35.764950 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1e018b-349b-4cfd-96d6-f81ca6d1de27 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:28:35.765439 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1e018b-349b-4cfd-96d6-f81ca6d1de27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:35.767102 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1e018b-349b-4cfd-96d6-f81ca6d1de27 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:35.780450 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 15/32] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:28:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:35.781772 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e018b-349b-4cfd-96d6-f81ca6d1de27 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:35.781899 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 18/33] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:28:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:28:35.786971 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd835e-5455-41bd-886d-249f2d456907 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:35.813313 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:35.893531 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e018b-349b-4cfd-96d6-f81ca6d1de27 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:35.899057 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd835e-5455-41bd-886d-249f2d456907 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:35.931302 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ba3b53-9361-41bf-b189-de8eaeb76b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:35.932146 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba3b53-9361-41bf-b189-de8eaeb76b30 None admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:35.932528 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba3b53-9361-41bf-b189-de8eaeb76b30 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:35.932900 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba3b53-9361-41bf-b189-de8eaeb76b30 None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:35.933956 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ba3b53-9361-41bf-b189-de8eaeb76b30 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:35.936038 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ba3b53-9361-41bf-b189-de8eaeb76b30 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:35.938423 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c8da4b-dad4-47b2-90ec-d7d860bfe4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:35.939193 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8da4b-dad4-47b2-90ec-d7d860bfe4fe None admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:35.939645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8da4b-dad4-47b2-90ec-d7d860bfe4fe None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:35.940052 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8da4b-dad4-47b2-90ec-d7d860bfe4fe None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:35.940988 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c8da4b-dad4-47b2-90ec-d7d860bfe4fe None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:35.942496 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c8da4b-dad4-47b2-90ec-d7d860bfe4fe None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:36.062213 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 16/34] 10.222.0.30 () {60 vars in 1131 bytes} [Tue Aug 30 14:28:35 2022] POST /identity/v3/roles => generated 219 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:28:36.068411 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ba3b53-9361-41bf-b189-de8eaeb76b30 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:36.071737 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 19/35] 10.222.0.30 () {60 vars in 1131 bytes} [Tue Aug 30 14:28:35 2022] POST /identity/v3/roles => generated 225 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:28:36.072939 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c8da4b-dad4-47b2-90ec-d7d860bfe4fe None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:36.127254 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:36.190274 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c8da4b-dad4-47b2-90ec-d7d860bfe4fe None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:36.243294 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c69961b-a7b7-424d-8168-e880a62f8baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:36.244317 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c69961b-a7b7-424d-8168-e880a62f8baf None admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:36.244823 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c69961b-a7b7-424d-8168-e880a62f8baf None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:36.245365 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c69961b-a7b7-424d-8168-e880a62f8baf None admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:36.246296 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c69961b-a7b7-424d-8168-e880a62f8baf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:36.249011 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c69961b-a7b7-424d-8168-e880a62f8baf None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:36.249625 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:36.274055 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ba3b53-9361-41bf-b189-de8eaeb76b30 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:36.298693 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 20/36] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Aug 30 14:28:36 2022] POST /identity/v3/projects => generated 301 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:28:36.338988 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a5cacd-de4f-4243-8ac8-4f06c5e8f936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:36.340040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a5cacd-de4f-4243-8ac8-4f06c5e8f936 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:36.340558 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a5cacd-de4f-4243-8ac8-4f06c5e8f936 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:36.341112 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a5cacd-de4f-4243-8ac8-4f06c5e8f936 None admin] PATH_INFO: `/v3/users/4a989d2c0a4a49eb9303e4db6cc0b7e8` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:36.344747 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a5cacd-de4f-4243-8ac8-4f06c5e8f936 None admin] RBAC: Authorizing `identity:get_user(user_id=4a989d2c0a4a49eb9303e4db6cc0b7e8)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:36.346842 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a5cacd-de4f-4243-8ac8-4f06c5e8f936 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:36.349028 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 17/37] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:28:36 2022] GET /identity/v3/users/4a989d2c0a4a49eb9303e4db6cc0b7e8 => generated 247 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:36.355017 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c69961b-a7b7-424d-8168-e880a62f8baf None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:36.438240 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a5943-d355-4df4-925d-588dea0fb954 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:36.439854 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a5943-d355-4df4-925d-588dea0fb954 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:36.440376 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a5943-d355-4df4-925d-588dea0fb954 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:36.442001 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 18/38] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:28:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:28:36.444347 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10388bcb-942c-43f3-9be6-a694b08d6b9d None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:36.444904 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10388bcb-942c-43f3-9be6-a694b08d6b9d None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:36.445370 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10388bcb-942c-43f3-9be6-a694b08d6b9d None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:36.447313 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 19/39] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:28:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:28:36.448770 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef68c2-1df3-42d2-8c1b-bb18fa44e3c3 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:36.449243 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef68c2-1df3-42d2-8c1b-bb18fa44e3c3 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:36.449805 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef68c2-1df3-42d2-8c1b-bb18fa44e3c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:36.477905 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-76ef68c2-1df3-42d2-8c1b-bb18fa44e3c3 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:28:36.480083 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-76ef68c2-1df3-42d2-8c1b-bb18fa44e3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:28:36.493327 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:36.498550 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c69961b-a7b7-424d-8168-e880a62f8baf None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:36.551151 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fbb0f1-6221-45bc-ba3c-f35d81aba174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:36.551151 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fbb0f1-6221-45bc-ba3c-f35d81aba174 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:36.551521 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fbb0f1-6221-45bc-ba3c-f35d81aba174 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:36.551743 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fbb0f1-6221-45bc-ba3c-f35d81aba174 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:36.553604 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fbb0f1-6221-45bc-ba3c-f35d81aba174 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:36.554241 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fbb0f1-6221-45bc-ba3c-f35d81aba174 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:36.555887 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 21/40] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:28:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:36.557126 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc1be1-c8e0-48ad-b9bc-60d4920271ed None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:36.557401 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc1be1-c8e0-48ad-b9bc-60d4920271ed None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:36.557681 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc1be1-c8e0-48ad-b9bc-60d4920271ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:36.576951 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 20/41] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:28:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:28:36.577623 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ef68c2-1df3-42d2-8c1b-bb18fa44e3c3 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:36.580593 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-afdc1be1-c8e0-48ad-b9bc-60d4920271ed None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:28:36.582157 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-afdc1be1-c8e0-48ad-b9bc-60d4920271ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:28:36.667318 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 22/42] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:28:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:28:36.676970 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:36.695446 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ef68c2-1df3-42d2-8c1b-bb18fa44e3c3 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:36.729988 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a452b10-20c2-4867-95ba-88332417e331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:36.730721 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a452b10-20c2-4867-95ba-88332417e331 None admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:36.730937 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a452b10-20c2-4867-95ba-88332417e331 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:36.731230 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a452b10-20c2-4867-95ba-88332417e331 None admin] PATH_INFO: `/v3/domains/default/users/4a989d2c0a4a49eb9303e4db6cc0b7e8/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:36.733946 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a452b10-20c2-4867-95ba-88332417e331 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=4a989d2c0a4a49eb9303e4db6cc0b7e8, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:36.735430 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a452b10-20c2-4867-95ba-88332417e331 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:36.758983 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 21/43] 10.222.0.30 () {58 vars in 1361 bytes} [Tue Aug 30 14:28:36 2022] PUT /identity/v3/domains/default/users/4a989d2c0a4a49eb9303e4db6cc0b7e8/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:28:36.846793 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdc1be1-c8e0-48ad-b9bc-60d4920271ed None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:36.937451 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a452b10-20c2-4867-95ba-88332417e331 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:37.049290 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a452b10-20c2-4867-95ba-88332417e331 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:37.058813 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:37.066063 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdc1be1-c8e0-48ad-b9bc-60d4920271ed None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:37.087723 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c570ae7-32e7-4cea-92c7-c51a4bebe841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:37.088362 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c570ae7-32e7-4cea-92c7-c51a4bebe841 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:37.088641 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c570ae7-32e7-4cea-92c7-c51a4bebe841 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:37.088911 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c570ae7-32e7-4cea-92c7-c51a4bebe841 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:37.090018 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c570ae7-32e7-4cea-92c7-c51a4bebe841 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:37.092025 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c570ae7-32e7-4cea-92c7-c51a4bebe841 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:37.093773 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 22/44] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:28:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:28:37.098956 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c570ae7-32e7-4cea-92c7-c51a4bebe841 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:37.104091 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca2fb44-0119-45f5-9c2d-302118b06105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:37.104711 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2fb44-0119-45f5-9c2d-302118b06105 None admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:37.104999 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2fb44-0119-45f5-9c2d-302118b06105 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:37.105276 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2fb44-0119-45f5-9c2d-302118b06105 None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:37.105952 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca2fb44-0119-45f5-9c2d-302118b06105 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:37.108023 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca2fb44-0119-45f5-9c2d-302118b06105 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:37.116001 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:37.123768 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 23/45] 10.222.0.30 () {60 vars in 1131 bytes} [Tue Aug 30 14:28:36 2022] POST /identity/v3/roles => generated 223 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:28:37.221002 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c570ae7-32e7-4cea-92c7-c51a4bebe841 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:37.257687 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd00f2c8-7e79-4ff3-a597-2b4754276d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:37.258314 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd00f2c8-7e79-4ff3-a597-2b4754276d31 None admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:37.258602 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd00f2c8-7e79-4ff3-a597-2b4754276d31 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:37.258878 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd00f2c8-7e79-4ff3-a597-2b4754276d31 None admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:37.259573 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd00f2c8-7e79-4ff3-a597-2b4754276d31 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:37.261046 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd00f2c8-7e79-4ff3-a597-2b4754276d31 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:37.294177 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 23/46] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Aug 30 14:28:37 2022] POST /identity/v3/projects => generated 312 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:28:38.690599 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9e714-199e-44e0-ac5f-24aaf5f7cc00 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:38.691109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9e714-199e-44e0-ac5f-24aaf5f7cc00 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:38.691255 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9e714-199e-44e0-ac5f-24aaf5f7cc00 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:38.692688 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 24/47] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:28:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:28:38.697575 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528874c-1c9a-4232-8f4f-24b1b64d4164 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:38.697962 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528874c-1c9a-4232-8f4f-24b1b64d4164 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:38.698268 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528874c-1c9a-4232-8f4f-24b1b64d4164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:38.716435 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-b528874c-1c9a-4232-8f4f-24b1b64d4164 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:28:38.717621 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-b528874c-1c9a-4232-8f4f-24b1b64d4164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:28:38.776474 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:38.787359 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 24/48] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:28:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:28:39.052390 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a9e714-199e-44e0-ac5f-24aaf5f7cc00 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:39.153868 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a9e714-199e-44e0-ac5f-24aaf5f7cc00 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:39.187064 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b8af85-22f2-4965-a900-f43569a3a08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:39.187744 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8af85-22f2-4965-a900-f43569a3a08c None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:39.188035 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8af85-22f2-4965-a900-f43569a3a08c None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:39.188318 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8af85-22f2-4965-a900-f43569a3a08c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:39.199005 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b8af85-22f2-4965-a900-f43569a3a08c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:28:39.203996 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-b2b8af85-22f2-4965-a900-f43569a3a08c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:28:39.203996 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:28:39.203996 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:28:39.203996 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:28:39.203996 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:28:39.203996 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:28:39.203996 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:28:39.203996 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:28:39.203996 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:28:39.203996 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:28:39.203996 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:28:39.203996 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:28:39.203996 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:28:39.203996 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:28:39.203996 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:28:39.203996 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:28:39.204817 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:28:39.204817 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:28:39.204817 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:28:39.204817 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:28:39.204817 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:28:39.204817 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:28:39.204817 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:28:39.204817 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:28:39.204817 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:28:39.204817 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:28:39.204817 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:28:39.204817 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:28:39.204817 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:28:39.204817 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:28:39.204817 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:28:39.204817 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:28:39.205557 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:28:39.205557 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:28:39.205557 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:28:39.205557 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:28:39.205557 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:28:39.205557 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:28:39.205557 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:28:39.205557 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:28:39.205557 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:28:39.205557 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:28:39.205557 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:28:39.205557 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:28:39.205557 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 25/49] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 30 14:28:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:28:39.210923 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b528874c-1c9a-4232-8f4f-24b1b64d4164 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:39.221092 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731e9ae-8489-47e8-9b62-1cd7ee860fa0 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:39.221392 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731e9ae-8489-47e8-9b62-1cd7ee860fa0 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:39.222061 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731e9ae-8489-47e8-9b62-1cd7ee860fa0 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:39.224044 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 26/50] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:28:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:28:39.228755 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af6ffe-5c72-4d60-b8bb-1f6d9b42003d None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:39.229054 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af6ffe-5c72-4d60-b8bb-1f6d9b42003d None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:39.229316 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af6ffe-5c72-4d60-b8bb-1f6d9b42003d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:39.248208 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-17af6ffe-5c72-4d60-b8bb-1f6d9b42003d None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:28:39.249416 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-17af6ffe-5c72-4d60-b8bb-1f6d9b42003d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:28:39.260087 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:39.313875 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b528874c-1c9a-4232-8f4f-24b1b64d4164 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:39.319347 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 27/51] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:28:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:28:39.347857 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14794783-fa6e-48bf-a70f-7071591314f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:39.348463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14794783-fa6e-48bf-a70f-7071591314f0 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:39.348744 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14794783-fa6e-48bf-a70f-7071591314f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:39.349016 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14794783-fa6e-48bf-a70f-7071591314f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:39.349675 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14794783-fa6e-48bf-a70f-7071591314f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:28:39.349975 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14794783-fa6e-48bf-a70f-7071591314f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:39.351395 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14794783-fa6e-48bf-a70f-7071591314f0 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:39.360045 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 25/52] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:28:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:39.365556 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af6ffe-5c72-4d60-b8bb-1f6d9b42003d None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:39.464714 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af6ffe-5c72-4d60-b8bb-1f6d9b42003d None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:39.498050 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e146308f-9014-49eb-b221-74c4933788db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:39.498677 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e146308f-9014-49eb-b221-74c4933788db None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:39.498962 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e146308f-9014-49eb-b221-74c4933788db None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:39.499270 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e146308f-9014-49eb-b221-74c4933788db None admin] PATH_INFO: `/v3/users/4a989d2c0a4a49eb9303e4db6cc0b7e8` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:39.500415 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e146308f-9014-49eb-b221-74c4933788db None admin] RBAC: Authorizing `identity:get_user(user_id=4a989d2c0a4a49eb9303e4db6cc0b7e8)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:39.501861 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e146308f-9014-49eb-b221-74c4933788db None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:39.503472 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 28/53] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:28:39 2022] GET /identity/v3/users/4a989d2c0a4a49eb9303e4db6cc0b7e8 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:39.509170 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14794783-fa6e-48bf-a70f-7071591314f0 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:39.592215 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e146308f-9014-49eb-b221-74c4933788db None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:39.609476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:39.673701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:39.695794 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e146308f-9014-49eb-b221-74c4933788db None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:39.718567 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14794783-fa6e-48bf-a70f-7071591314f0 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:39.730085 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3845280-0f52-47a6-ad7f-795cd4fb1ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:39.730748 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3845280-0f52-47a6-ad7f-795cd4fb1ccf None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:39.731005 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3845280-0f52-47a6-ad7f-795cd4fb1ccf None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:39.731305 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3845280-0f52-47a6-ad7f-795cd4fb1ccf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:39.733399 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3845280-0f52-47a6-ad7f-795cd4fb1ccf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:39.734873 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3845280-0f52-47a6-ad7f-795cd4fb1ccf None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:39.736647 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 29/54] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:28:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:28:39.742099 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3845280-0f52-47a6-ad7f-795cd4fb1ccf None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:39.753441 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6204b52-02f3-441f-a2b4-fbea99a398f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:39.754085 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6204b52-02f3-441f-a2b4-fbea99a398f2 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:39.754372 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6204b52-02f3-441f-a2b4-fbea99a398f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:39.754650 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6204b52-02f3-441f-a2b4-fbea99a398f2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:39.755805 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6204b52-02f3-441f-a2b4-fbea99a398f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:39.757270 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6204b52-02f3-441f-a2b4-fbea99a398f2 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:39.758857 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 26/55] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:28:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:39.764581 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6204b52-02f3-441f-a2b4-fbea99a398f2 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:39.824513 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:39.844698 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3845280-0f52-47a6-ad7f-795cd4fb1ccf None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:39.866481 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6204b52-02f3-441f-a2b4-fbea99a398f2 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:39.879019 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b7aaf-61e9-4dad-9fbe-109f4b8d2ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:39.879661 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b7aaf-61e9-4dad-9fbe-109f4b8d2ac8 None admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:39.879950 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b7aaf-61e9-4dad-9fbe-109f4b8d2ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:39.880228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b7aaf-61e9-4dad-9fbe-109f4b8d2ac8 None admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:39.880881 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1b7aaf-61e9-4dad-9fbe-109f4b8d2ac8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:39.882324 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1b7aaf-61e9-4dad-9fbe-109f4b8d2ac8 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:39.891716 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:39.900619 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b468fb9-7906-4a6b-9860-bbc59585a1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:39.901233 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b468fb9-7906-4a6b-9860-bbc59585a1f8 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:39.901593 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b468fb9-7906-4a6b-9860-bbc59585a1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:39.901903 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b468fb9-7906-4a6b-9860-bbc59585a1f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:39.902610 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b468fb9-7906-4a6b-9860-bbc59585a1f8 None admin] RBAC: Adding query filter params (role.id=3cda0615d87d4af0860a2a5e9b4e2c05, scope.domain.id=default, user.id=4a989d2c0a4a49eb9303e4db6cc0b7e8) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:28:39.902912 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b468fb9-7906-4a6b-9860-bbc59585a1f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:39.905052 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b468fb9-7906-4a6b-9860-bbc59585a1f8 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:39.908768 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 30/56] 10.222.0.30 () {60 vars in 1140 bytes} [Tue Aug 30 14:28:39 2022] POST /identity/v3/projects => generated 298 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:28:39.913698 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 27/57] 10.222.0.30 () {56 vars in 1322 bytes} [Tue Aug 30 14:28:39 2022] GET /identity/v3/role_assignments?user.id=4a989d2c0a4a49eb9303e4db6cc0b7e8&scope.domain.id=default&role.id=3cda0615d87d4af0860a2a5e9b4e2c05 => generated 530 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:28:41.817742 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c252412-a44c-4206-ae43-dc3b6b854cac None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:41.818236 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c252412-a44c-4206-ae43-dc3b6b854cac None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:41.818322 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c252412-a44c-4206-ae43-dc3b6b854cac None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:41.819665 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 31/58] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:28:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:28:41.824591 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50679ffc-9baf-457d-b757-8da4b588260f None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:41.824948 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50679ffc-9baf-457d-b757-8da4b588260f None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:41.825145 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50679ffc-9baf-457d-b757-8da4b588260f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:41.844012 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-50679ffc-9baf-457d-b757-8da4b588260f None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:28:41.845196 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-50679ffc-9baf-457d-b757-8da4b588260f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:28:41.909180 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 28/59] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:28:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:28:42.171918 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c252412-a44c-4206-ae43-dc3b6b854cac None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:42.275697 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c252412-a44c-4206-ae43-dc3b6b854cac None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:42.313193 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d7a833-8767-41b0-9515-1ba22b58ba6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:42.313808 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d7a833-8767-41b0-9515-1ba22b58ba6d None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:42.314091 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d7a833-8767-41b0-9515-1ba22b58ba6d None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:42.314365 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d7a833-8767-41b0-9515-1ba22b58ba6d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:42.315501 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d7a833-8767-41b0-9515-1ba22b58ba6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:42.316933 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d7a833-8767-41b0-9515-1ba22b58ba6d None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:42.318491 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 32/60] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:28:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:42.323971 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50679ffc-9baf-457d-b757-8da4b588260f None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:42.418966 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50679ffc-9baf-457d-b757-8da4b588260f None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:42.438583 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:42.452776 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bfa9f1-7b78-45fc-909f-b114e0306954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:42.453374 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bfa9f1-7b78-45fc-909f-b114e0306954 None admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:42.453651 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bfa9f1-7b78-45fc-909f-b114e0306954 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:42.453917 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bfa9f1-7b78-45fc-909f-b114e0306954 None admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:42.454553 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bfa9f1-7b78-45fc-909f-b114e0306954 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:42.456031 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bfa9f1-7b78-45fc-909f-b114e0306954 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:42.480892 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 29/61] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 30 14:28:42 2022] POST /identity/v3/users => generated 275 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:28:45.289660 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9532138-2332-430b-a042-56503e5b4c5e None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:45.290687 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9532138-2332-430b-a042-56503e5b4c5e None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:45.291165 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9532138-2332-430b-a042-56503e5b4c5e None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:45.292858 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 33/62] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:28:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:28:45.298901 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf5522-a38f-4ea4-b68c-8fd51b48e04a None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:45.299324 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf5522-a38f-4ea4-b68c-8fd51b48e04a None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:45.299497 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf5522-a38f-4ea4-b68c-8fd51b48e04a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:45.323441 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-edcf5522-a38f-4ea4-b68c-8fd51b48e04a None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:28:45.324824 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-edcf5522-a38f-4ea4-b68c-8fd51b48e04a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:28:45.394390 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 30/63] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:28:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:28:45.555562 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edbf08f-6e8b-4d29-8159-8eb588520c85 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:45.555907 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edbf08f-6e8b-4d29-8159-8eb588520c85 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:45.556134 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edbf08f-6e8b-4d29-8159-8eb588520c85 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:45.557481 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 34/64] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:28:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:28:45.562904 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9a7cc-fa29-492a-bd56-1c32998ebe63 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:45.563487 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9a7cc-fa29-492a-bd56-1c32998ebe63 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:45.563955 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9a7cc-fa29-492a-bd56-1c32998ebe63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:45.592524 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-f0d9a7cc-fa29-492a-bd56-1c32998ebe63 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:28:45.594043 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-f0d9a7cc-fa29-492a-bd56-1c32998ebe63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:28:45.684163 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 31/65] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:28:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:28:45.757539 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edbf08f-6e8b-4d29-8159-8eb588520c85 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:45.760783 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:45.888598 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edbf08f-6e8b-4d29-8159-8eb588520c85 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:45.891777 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef165214-5629-46e5-9819-9731abaf3c18 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:45.892275 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef165214-5629-46e5-9819-9731abaf3c18 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:45.892766 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef165214-5629-46e5-9819-9731abaf3c18 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:45.894297 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 32/66] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:28:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:28:45.899371 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a63d2-ab05-419e-b3f0-1ebf9c3323e2 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:45.899886 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a63d2-ab05-419e-b3f0-1ebf9c3323e2 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:45.900355 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a63d2-ab05-419e-b3f0-1ebf9c3323e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:45.918829 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-227a63d2-ab05-419e-b3f0-1ebf9c3323e2 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:28:45.920822 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-227a63d2-ab05-419e-b3f0-1ebf9c3323e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:28:45.949266 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc47567-84f8-406a-8c92-8dc06f7d20d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:45.950219 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc47567-84f8-406a-8c92-8dc06f7d20d2 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:45.951526 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc47567-84f8-406a-8c92-8dc06f7d20d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:45.952038 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc47567-84f8-406a-8c92-8dc06f7d20d2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:45.965659 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:45.973049 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc47567-84f8-406a-8c92-8dc06f7d20d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:28:45.978459 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-ddc47567-84f8-406a-8c92-8dc06f7d20d2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:28:45.978459 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:28:45.978459 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:28:45.978459 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:28:45.978459 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:28:45.978459 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:28:45.978459 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:28:45.978459 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:28:45.978459 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:28:45.978459 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:28:45.978459 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:28:45.978459 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:28:45.978459 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:28:45.978459 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:28:45.978459 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:28:45.978459 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:28:45.979394 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:28:45.979394 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:28:45.979394 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:28:45.979394 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:28:45.979394 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:28:45.979394 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:28:45.979394 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:28:45.979394 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:28:45.979394 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:28:45.979394 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:28:45.979394 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:28:45.979394 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:28:45.979394 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:28:45.979394 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:28:45.979394 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:28:45.979394 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:28:45.980221 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:28:45.980221 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:28:45.980221 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:28:45.980221 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:28:45.980221 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:28:45.980221 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:28:45.980221 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:28:45.980221 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:28:45.980221 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:28:45.980221 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:28:45.980221 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:28:45.980221 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:28:45.980221 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 35/67] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:28:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:28:45.988206 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc47567-84f8-406a-8c92-8dc06f7d20d2 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:46.020458 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 33/68] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:28:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:28:46.062169 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:46.062852 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a63d2-ab05-419e-b3f0-1ebf9c3323e2 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:46.123559 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc47567-84f8-406a-8c92-8dc06f7d20d2 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:46.175597 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a63d2-ab05-419e-b3f0-1ebf9c3323e2 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:46.187938 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fccaf3-61f5-43ba-a88b-c72fdb86aed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:46.188592 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fccaf3-61f5-43ba-a88b-c72fdb86aed0 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:46.189026 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fccaf3-61f5-43ba-a88b-c72fdb86aed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:46.189129 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fccaf3-61f5-43ba-a88b-c72fdb86aed0 None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:46.189807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fccaf3-61f5-43ba-a88b-c72fdb86aed0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:28:46.190120 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fccaf3-61f5-43ba-a88b-c72fdb86aed0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:46.191548 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fccaf3-61f5-43ba-a88b-c72fdb86aed0 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:46.204653 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 36/69] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 30 14:28:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:46.213117 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fccaf3-61f5-43ba-a88b-c72fdb86aed0 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:46.218186 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4fc2e0-8e99-4dc5-bb94-7d71cea103bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:46.218186 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4fc2e0-8e99-4dc5-bb94-7d71cea103bf None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:46.218186 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4fc2e0-8e99-4dc5-bb94-7d71cea103bf None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:46.218186 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4fc2e0-8e99-4dc5-bb94-7d71cea103bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:46.229049 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4fc2e0-8e99-4dc5-bb94-7d71cea103bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:28:46.235182 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-ef4fc2e0-8e99-4dc5-bb94-7d71cea103bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:28:46.235182 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:28:46.235182 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:28:46.235182 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:28:46.235182 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:28:46.235182 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:28:46.235182 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:28:46.235182 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:28:46.235182 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:28:46.235182 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:28:46.235182 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:28:46.235182 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:28:46.235182 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:28:46.235182 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:28:46.235182 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:28:46.235182 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:28:46.236157 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:28:46.236157 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:28:46.236157 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:28:46.236157 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:28:46.236157 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:28:46.236157 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:28:46.236157 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:28:46.236157 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:28:46.236157 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:28:46.236157 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:28:46.236157 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:28:46.236157 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:28:46.236157 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:28:46.236157 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:28:46.236157 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:28:46.236157 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:28:46.236911 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:28:46.236911 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:28:46.236911 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:28:46.236911 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:28:46.236911 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:28:46.236911 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:28:46.236911 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:28:46.236911 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:28:46.236911 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:28:46.236911 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:28:46.236911 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:28:46.236911 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:28:46.236911 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 34/70] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 30 14:28:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:28:46.238873 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e458b-4c85-4815-8fcc-8d298389405b None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:46.239238 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e458b-4c85-4815-8fcc-8d298389405b None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:46.239611 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e458b-4c85-4815-8fcc-8d298389405b None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:46.241203 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 35/71] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:28:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:28:46.249265 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2b165-ebe9-4885-8a10-4c7b3e5dfb5f None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:46.249787 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2b165-ebe9-4885-8a10-4c7b3e5dfb5f None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:46.250341 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2b165-ebe9-4885-8a10-4c7b3e5dfb5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:46.271403 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-b6d2b165-ebe9-4885-8a10-4c7b3e5dfb5f None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:28:46.272856 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-b6d2b165-ebe9-4885-8a10-4c7b3e5dfb5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:28:46.298815 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:46.329483 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fccaf3-61f5-43ba-a88b-c72fdb86aed0 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:46.350403 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 36/72] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:28:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:28:46.351298 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d2b165-ebe9-4885-8a10-4c7b3e5dfb5f None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:46.362507 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:46.374665 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04e5d0b-f681-4f02-838a-3cbe95dece80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:46.375342 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e5d0b-f681-4f02-838a-3cbe95dece80 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:46.375633 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e5d0b-f681-4f02-838a-3cbe95dece80 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:46.375919 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e5d0b-f681-4f02-838a-3cbe95dece80 None admin] PATH_INFO: `/v3/users/873ad929e6dd4a1c9f5875c6017d0fe0` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:46.390239 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04e5d0b-f681-4f02-838a-3cbe95dece80 None admin] RBAC: Authorizing `identity:get_user(user_id=873ad929e6dd4a1c9f5875c6017d0fe0)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:46.391751 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04e5d0b-f681-4f02-838a-3cbe95dece80 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:46.393595 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 37/73] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:28:46 2022] GET /identity/v3/users/873ad929e6dd4a1c9f5875c6017d0fe0 => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:46.394749 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c51ad-15e2-4219-b6d6-eab13232b273 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:46.395045 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c51ad-15e2-4219-b6d6-eab13232b273 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:46.395356 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c51ad-15e2-4219-b6d6-eab13232b273 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:46.396692 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 38/74] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:28:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:28:46.400120 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51c51ad-15e2-4219-b6d6-eab13232b273 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:46.463715 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d2b165-ebe9-4885-8a10-4c7b3e5dfb5f None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:46.499511 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfce488-d7b6-4bdc-a5b0-128ca188aed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:46.500121 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfce488-d7b6-4bdc-a5b0-128ca188aed0 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:46.500407 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfce488-d7b6-4bdc-a5b0-128ca188aed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:46.500663 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfce488-d7b6-4bdc-a5b0-128ca188aed0 None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:46.501325 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfce488-d7b6-4bdc-a5b0-128ca188aed0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:28:46.501789 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfce488-d7b6-4bdc-a5b0-128ca188aed0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:46.503251 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfce488-d7b6-4bdc-a5b0-128ca188aed0 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:46.517424 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 37/75] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:28:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:46.517570 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51c51ad-15e2-4219-b6d6-eab13232b273 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:46.517661 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfce488-d7b6-4bdc-a5b0-128ca188aed0 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:46.537814 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:46.553450 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5812185e-bd53-4a73-9a7f-319229474bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:46.554090 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5812185e-bd53-4a73-9a7f-319229474bb7 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:46.554470 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5812185e-bd53-4a73-9a7f-319229474bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:46.554647 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5812185e-bd53-4a73-9a7f-319229474bb7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:46.565347 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5812185e-bd53-4a73-9a7f-319229474bb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:28:46.570182 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-5812185e-bd53-4a73-9a7f-319229474bb7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:28:46.570182 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:28:46.570182 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:28:46.570182 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:28:46.570182 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:28:46.570182 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:28:46.570182 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:28:46.570182 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:28:46.570182 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:28:46.570182 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:28:46.570182 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:28:46.570182 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:28:46.570182 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:28:46.570182 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:28:46.570182 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:28:46.570182 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:28:46.571012 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:28:46.571012 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:28:46.571012 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:28:46.571012 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:28:46.571012 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:28:46.571012 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:28:46.571012 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:28:46.571012 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:28:46.571012 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:28:46.571012 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:28:46.571012 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:28:46.571012 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:28:46.571012 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:28:46.571012 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:28:46.571012 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:28:46.571012 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:28:46.571795 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:28:46.571795 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:28:46.571795 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:28:46.571795 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:28:46.571795 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:28:46.571795 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:28:46.571795 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:28:46.571795 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:28:46.571795 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:28:46.571795 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:28:46.571795 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:28:46.571795 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:28:46.571795 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 39/76] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 30 14:28:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:28:46.572821 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af05094-a27b-4cf8-bf00-0e65c07fc9bc None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:46.572821 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af05094-a27b-4cf8-bf00-0e65c07fc9bc None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:46.572995 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af05094-a27b-4cf8-bf00-0e65c07fc9bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:46.591258 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-4af05094-a27b-4cf8-bf00-0e65c07fc9bc None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:28:46.592684 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-4af05094-a27b-4cf8-bf00-0e65c07fc9bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:28:46.628391 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfce488-d7b6-4bdc-a5b0-128ca188aed0 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:46.635310 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:46.661873 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c5705-3227-47eb-806f-551f7a2df4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:46.662485 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c5705-3227-47eb-806f-551f7a2df4c8 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:46.662781 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c5705-3227-47eb-806f-551f7a2df4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:46.663036 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c5705-3227-47eb-806f-551f7a2df4c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:46.668225 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 40/77] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:28:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:28:46.669617 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af05094-a27b-4cf8-bf00-0e65c07fc9bc None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:46.742033 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c5705-3227-47eb-806f-551f7a2df4c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:46.743481 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c5705-3227-47eb-806f-551f7a2df4c8 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:46.762371 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:46.770737 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af05094-a27b-4cf8-bf00-0e65c07fc9bc None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:46.806220 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aee70c-b33a-4618-9ae5-9137e7c0d64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:46.806853 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aee70c-b33a-4618-9ae5-9137e7c0d64f None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:46.807134 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aee70c-b33a-4618-9ae5-9137e7c0d64f None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:46.807434 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aee70c-b33a-4618-9ae5-9137e7c0d64f None admin] PATH_INFO: `/v3/users/4a989d2c0a4a49eb9303e4db6cc0b7e8` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:46.808845 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aee70c-b33a-4618-9ae5-9137e7c0d64f None admin] RBAC: Authorizing `identity:get_user(user_id=4a989d2c0a4a49eb9303e4db6cc0b7e8)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:46.810797 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aee70c-b33a-4618-9ae5-9137e7c0d64f None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:46.812420 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 41/78] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:28:46 2022] GET /identity/v3/users/4a989d2c0a4a49eb9303e4db6cc0b7e8 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:46.813172 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aee70c-b33a-4618-9ae5-9137e7c0d64f None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:46.859453 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:46.862432 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 38/79] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:28:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:28:46.867592 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c5705-3227-47eb-806f-551f7a2df4c8 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:46.921740 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aee70c-b33a-4618-9ae5-9137e7c0d64f None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:46.958650 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd646fe-0928-450a-afd2-3b2a1c0384da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:46.959325 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd646fe-0928-450a-afd2-3b2a1c0384da None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:46.959612 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd646fe-0928-450a-afd2-3b2a1c0384da None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:46.959887 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd646fe-0928-450a-afd2-3b2a1c0384da None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:46.960552 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd646fe-0928-450a-afd2-3b2a1c0384da None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:28:46.960854 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd646fe-0928-450a-afd2-3b2a1c0384da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:46.962227 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd646fe-0928-450a-afd2-3b2a1c0384da None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:46.970048 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 42/80] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:28:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:46.970841 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd646fe-0928-450a-afd2-3b2a1c0384da None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:46.972462 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c5705-3227-47eb-806f-551f7a2df4c8 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:46.990265 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:47.007075 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fccc002-f856-44ce-8225-6482525eea9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:47.007685 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fccc002-f856-44ce-8225-6482525eea9a None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:47.007966 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fccc002-f856-44ce-8225-6482525eea9a None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:47.008239 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fccc002-f856-44ce-8225-6482525eea9a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:47.019178 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fccc002-f856-44ce-8225-6482525eea9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:28:47.024003 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-7fccc002-f856-44ce-8225-6482525eea9a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:28:47.024003 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:28:47.024003 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:28:47.024003 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:28:47.024003 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:28:47.024003 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:28:47.024003 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:28:47.024003 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:28:47.024003 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:28:47.024003 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:28:47.024003 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:28:47.024003 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:28:47.024003 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:28:47.024003 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:28:47.024003 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:28:47.024003 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:28:47.025049 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:28:47.025049 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:28:47.025049 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:28:47.025049 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:28:47.025049 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:28:47.025049 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:28:47.025049 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:28:47.025049 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:28:47.025049 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:28:47.025049 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:28:47.025049 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:28:47.025049 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:28:47.025049 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:28:47.025049 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:28:47.025049 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:28:47.025049 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:28:47.025798 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:28:47.025798 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:28:47.025798 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:28:47.025798 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:28:47.025798 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:28:47.025798 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:28:47.025798 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:28:47.025798 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:28:47.025798 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:28:47.025798 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:28:47.025798 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:28:47.025798 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:28:47.025798 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 39/81] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:28:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:28:47.026452 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fccc002-f856-44ce-8225-6482525eea9a None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:47.070733 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd646fe-0928-450a-afd2-3b2a1c0384da None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:47.072278 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:47.105073 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9215a45-c645-455d-8b59-42995a063ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:47.105683 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9215a45-c645-455d-8b59-42995a063ad1 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:47.105971 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9215a45-c645-455d-8b59-42995a063ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:47.106248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9215a45-c645-455d-8b59-42995a063ad1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:47.125550 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fccc002-f856-44ce-8225-6482525eea9a None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:47.160956 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832bcfd2-1c0a-4418-821f-9dd21c0e2b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:47.161586 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832bcfd2-1c0a-4418-821f-9dd21c0e2b27 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:47.161870 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832bcfd2-1c0a-4418-821f-9dd21c0e2b27 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:47.162138 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832bcfd2-1c0a-4418-821f-9dd21c0e2b27 None admin] PATH_INFO: `/v3/projects/1c2139beef6b44b9a0574047298c1af9` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:47.163314 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832bcfd2-1c0a-4418-821f-9dd21c0e2b27 None admin] RBAC: Authorizing `identity:get_project(project_id=1c2139beef6b44b9a0574047298c1af9)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:47.164774 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832bcfd2-1c0a-4418-821f-9dd21c0e2b27 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:47.171402 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 40/82] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:28:47 2022] GET /identity/v3/projects/1c2139beef6b44b9a0574047298c1af9 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:47.171402 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832bcfd2-1c0a-4418-821f-9dd21c0e2b27 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:47.176044 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9215a45-c645-455d-8b59-42995a063ad1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:47.177518 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9215a45-c645-455d-8b59-42995a063ad1 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:47.213268 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:47.266835 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832bcfd2-1c0a-4418-821f-9dd21c0e2b27 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:47.280735 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 43/83] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:28:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:28:47.280735 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9215a45-c645-455d-8b59-42995a063ad1 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:47.280735 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:47.305941 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dac2e96-188e-42f8-bdf3-98d0e8f73de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:47.306573 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dac2e96-188e-42f8-bdf3-98d0e8f73de0 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:47.306855 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dac2e96-188e-42f8-bdf3-98d0e8f73de0 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:47.307127 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dac2e96-188e-42f8-bdf3-98d0e8f73de0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:47.308527 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dac2e96-188e-42f8-bdf3-98d0e8f73de0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:47.310205 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dac2e96-188e-42f8-bdf3-98d0e8f73de0 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:47.312112 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 41/84] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:28:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:47.312788 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dac2e96-188e-42f8-bdf3-98d0e8f73de0 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:47.385476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9215a45-c645-455d-8b59-42995a063ad1 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:47.413298 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dac2e96-188e-42f8-bdf3-98d0e8f73de0 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:47.418337 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2be65cc-2eda-4c63-936a-403d080afbce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:47.418965 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be65cc-2eda-4c63-936a-403d080afbce None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:47.419300 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be65cc-2eda-4c63-936a-403d080afbce None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:47.419576 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be65cc-2eda-4c63-936a-403d080afbce None admin] PATH_INFO: `/v3/users/873ad929e6dd4a1c9f5875c6017d0fe0` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:47.420707 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2be65cc-2eda-4c63-936a-403d080afbce None admin] RBAC: Authorizing `identity:get_user(user_id=873ad929e6dd4a1c9f5875c6017d0fe0)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:47.422159 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2be65cc-2eda-4c63-936a-403d080afbce None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:47.428463 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 44/85] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:28:47 2022] GET /identity/v3/users/873ad929e6dd4a1c9f5875c6017d0fe0 => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:47.428463 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2be65cc-2eda-4c63-936a-403d080afbce None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:47.428683 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:47.448162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de45836-5368-4b32-a694-8761210c47cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:47.448774 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de45836-5368-4b32-a694-8761210c47cd None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:47.449060 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de45836-5368-4b32-a694-8761210c47cd None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:47.449335 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de45836-5368-4b32-a694-8761210c47cd None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:47.449999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de45836-5368-4b32-a694-8761210c47cd None admin] RBAC: Adding query filter params (name=member) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:28:47.450937 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de45836-5368-4b32-a694-8761210c47cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:47.452475 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de45836-5368-4b32-a694-8761210c47cd None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:47.460504 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 42/86] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 30 14:28:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:47.463364 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de45836-5368-4b32-a694-8761210c47cd None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:47.502649 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:47.534058 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2be65cc-2eda-4c63-936a-403d080afbce None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:47.562390 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de45836-5368-4b32-a694-8761210c47cd None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:47.571019 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46204965-1621-44f1-95c7-611d6cc8c436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:47.571657 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46204965-1621-44f1-95c7-611d6cc8c436 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:47.571936 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46204965-1621-44f1-95c7-611d6cc8c436 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:47.572207 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46204965-1621-44f1-95c7-611d6cc8c436 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:47.572934 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46204965-1621-44f1-95c7-611d6cc8c436 None admin] RBAC: Adding query filter params (role.id=6a985d5659a3416cba33ca9d30b39f85, scope.project.id=1c2139beef6b44b9a0574047298c1af9, user.id=873ad929e6dd4a1c9f5875c6017d0fe0) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:28:47.573245 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46204965-1621-44f1-95c7-611d6cc8c436 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:47.574679 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46204965-1621-44f1-95c7-611d6cc8c436 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:47.580260 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 45/87] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:28:47 2022] GET /identity/v3/role_assignments?user.id=873ad929e6dd4a1c9f5875c6017d0fe0&scope.project.id=1c2139beef6b44b9a0574047298c1af9&role.id=6a985d5659a3416cba33ca9d30b39f85 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:47.581030 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46204965-1621-44f1-95c7-611d6cc8c436 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:47.600155 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3addc499-8ed4-4535-b486-2704eafec864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:47.600818 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3addc499-8ed4-4535-b486-2704eafec864 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:47.601099 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3addc499-8ed4-4535-b486-2704eafec864 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:47.601373 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3addc499-8ed4-4535-b486-2704eafec864 None admin] PATH_INFO: `/v3/projects/1c2139beef6b44b9a0574047298c1af9` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:47.602625 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3addc499-8ed4-4535-b486-2704eafec864 None admin] RBAC: Authorizing `identity:get_project(project_id=1c2139beef6b44b9a0574047298c1af9)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:47.604152 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3addc499-8ed4-4535-b486-2704eafec864 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:47.611829 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 43/88] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:28:47 2022] GET /identity/v3/projects/1c2139beef6b44b9a0574047298c1af9 => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:47.611829 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3addc499-8ed4-4535-b486-2704eafec864 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:47.665353 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:47.711874 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46204965-1621-44f1-95c7-611d6cc8c436 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:47.730626 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3addc499-8ed4-4535-b486-2704eafec864 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:47.754470 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1743f8fe-dc68-4971-9cd5-456467dcbe2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:47.755088 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1743f8fe-dc68-4971-9cd5-456467dcbe2f None admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:47.755398 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1743f8fe-dc68-4971-9cd5-456467dcbe2f None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:47.755671 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1743f8fe-dc68-4971-9cd5-456467dcbe2f None admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:47.756324 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1743f8fe-dc68-4971-9cd5-456467dcbe2f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:47.757779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1743f8fe-dc68-4971-9cd5-456467dcbe2f None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:47.768031 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a0843-ffc1-42a8-80ee-9073dfa1b6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:47.768689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:47.769006 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a0843-ffc1-42a8-80ee-9073dfa1b6b3 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:47.769294 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a0843-ffc1-42a8-80ee-9073dfa1b6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:47.769575 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a0843-ffc1-42a8-80ee-9073dfa1b6b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:47.786673 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 46/89] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 30 14:28:47 2022] POST /identity/v3/users => generated 289 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:28:47.788303 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1743f8fe-dc68-4971-9cd5-456467dcbe2f None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:47.862875 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4a0843-ffc1-42a8-80ee-9073dfa1b6b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:47.864852 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4a0843-ffc1-42a8-80ee-9073dfa1b6b3 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:47.927741 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:47.939689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1743f8fe-dc68-4971-9cd5-456467dcbe2f None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:47.980606 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2589bdac-ed51-4003-9683-ffa0f12ef593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:47.981801 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589bdac-ed51-4003-9683-ffa0f12ef593 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:47.982211 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589bdac-ed51-4003-9683-ffa0f12ef593 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:47.982532 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589bdac-ed51-4003-9683-ffa0f12ef593 None admin] PATH_INFO: `/v3/users/873ad929e6dd4a1c9f5875c6017d0fe0` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:47.983975 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2589bdac-ed51-4003-9683-ffa0f12ef593 None admin] RBAC: Authorizing `identity:get_user(user_id=873ad929e6dd4a1c9f5875c6017d0fe0)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:47.985992 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2589bdac-ed51-4003-9683-ffa0f12ef593 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:47.987934 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 47/90] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:28:47 2022] GET /identity/v3/users/873ad929e6dd4a1c9f5875c6017d0fe0 => generated 275 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:47.988461 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2589bdac-ed51-4003-9683-ffa0f12ef593 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:48.004135 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 44/91] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:28:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:28:48.004998 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a0843-ffc1-42a8-80ee-9073dfa1b6b3 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:48.083154 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:48.132978 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a0843-ffc1-42a8-80ee-9073dfa1b6b3 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:48.134697 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2589bdac-ed51-4003-9683-ffa0f12ef593 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:48.177288 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe11a79-bd0a-4c25-85d4-e8e23c6d0922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:48.177907 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe11a79-bd0a-4c25-85d4-e8e23c6d0922 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:48.178193 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe11a79-bd0a-4c25-85d4-e8e23c6d0922 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:48.178472 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe11a79-bd0a-4c25-85d4-e8e23c6d0922 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:48.179396 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af970526-4e57-4af0-9ffa-f1702fb26de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:48.180103 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af970526-4e57-4af0-9ffa-f1702fb26de7 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:48.180456 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af970526-4e57-4af0-9ffa-f1702fb26de7 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:48.180799 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af970526-4e57-4af0-9ffa-f1702fb26de7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:48.181600 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af970526-4e57-4af0-9ffa-f1702fb26de7 None admin] RBAC: Adding query filter params (role.id=3cda0615d87d4af0860a2a5e9b4e2c05, scope.project.id=1c2139beef6b44b9a0574047298c1af9, user.id=4a989d2c0a4a49eb9303e4db6cc0b7e8) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:28:48.181982 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af970526-4e57-4af0-9ffa-f1702fb26de7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:48.183804 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af970526-4e57-4af0-9ffa-f1702fb26de7 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:48.190182 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 48/92] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:28:47 2022] GET /identity/v3/role_assignments?user.id=4a989d2c0a4a49eb9303e4db6cc0b7e8&scope.project.id=1c2139beef6b44b9a0574047298c1af9&role.id=3cda0615d87d4af0860a2a5e9b4e2c05 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:48.190834 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af970526-4e57-4af0-9ffa-f1702fb26de7 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:48.195756 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:48.333373 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af970526-4e57-4af0-9ffa-f1702fb26de7 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:48.360388 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe11a79-bd0a-4c25-85d4-e8e23c6d0922 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:48.361919 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe11a79-bd0a-4c25-85d4-e8e23c6d0922 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:48.362949 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:48.386067 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670d09f4-d61e-4431-a0f1-26d4c3670c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:48.387510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670d09f4-d61e-4431-a0f1-26d4c3670c52 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:48.388547 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670d09f4-d61e-4431-a0f1-26d4c3670c52 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:48.389037 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670d09f4-d61e-4431-a0f1-26d4c3670c52 None admin] PATH_INFO: `/v3/projects/1c2139beef6b44b9a0574047298c1af9` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:48.392176 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670d09f4-d61e-4431-a0f1-26d4c3670c52 None admin] RBAC: Authorizing `identity:get_project(project_id=1c2139beef6b44b9a0574047298c1af9)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:48.393877 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670d09f4-d61e-4431-a0f1-26d4c3670c52 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:48.399604 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 49/93] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:28:48 2022] GET /identity/v3/projects/1c2139beef6b44b9a0574047298c1af9 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:48.404664 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670d09f4-d61e-4431-a0f1-26d4c3670c52 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:48.497116 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 45/94] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:28:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:28:48.503821 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe11a79-bd0a-4c25-85d4-e8e23c6d0922 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:48.607197 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:48.613190 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670d09f4-d61e-4431-a0f1-26d4c3670c52 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:48.649449 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a348da1-33cc-4758-992c-a11451c87c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:48.650068 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a348da1-33cc-4758-992c-a11451c87c93 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:48.650356 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a348da1-33cc-4758-992c-a11451c87c93 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:48.650632 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a348da1-33cc-4758-992c-a11451c87c93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:48.651712 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a348da1-33cc-4758-992c-a11451c87c93 None admin] RBAC: Adding query filter params (role.id=1c4886a7a7fd402d8221a0158ddcb167, scope.project.id=1c2139beef6b44b9a0574047298c1af9, user.id=873ad929e6dd4a1c9f5875c6017d0fe0) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:28:48.652027 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a348da1-33cc-4758-992c-a11451c87c93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:48.653713 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a348da1-33cc-4758-992c-a11451c87c93 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:48.663234 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 50/95] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:28:48 2022] GET /identity/v3/role_assignments?user.id=873ad929e6dd4a1c9f5875c6017d0fe0&scope.project.id=1c2139beef6b44b9a0574047298c1af9&role.id=1c4886a7a7fd402d8221a0158ddcb167 => generated 259 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:48.666310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe11a79-bd0a-4c25-85d4-e8e23c6d0922 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:48.704296 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46c2cdf-881d-4b66-bfa3-043b5dde0b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:48.704946 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c2cdf-881d-4b66-bfa3-043b5dde0b69 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:48.705229 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c2cdf-881d-4b66-bfa3-043b5dde0b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:48.705505 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c2cdf-881d-4b66-bfa3-043b5dde0b69 None admin] PATH_INFO: `/v3/projects/e319c5f287b64c0d824e38968057b3b7` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:48.706678 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46c2cdf-881d-4b66-bfa3-043b5dde0b69 None admin] RBAC: Authorizing `identity:get_project(project_id=e319c5f287b64c0d824e38968057b3b7)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:48.708170 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46c2cdf-881d-4b66-bfa3-043b5dde0b69 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:48.710000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 46/96] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:28:48 2022] GET /identity/v3/projects/e319c5f287b64c0d824e38968057b3b7 => generated 312 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:48.716589 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a348da1-33cc-4758-992c-a11451c87c93 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:48.779910 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:48.830212 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a348da1-33cc-4758-992c-a11451c87c93 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:48.870020 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3033fb-b046-4423-90c5-0767eb8e4e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:48.870572 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3033fb-b046-4423-90c5-0767eb8e4e4b None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:48.870861 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3033fb-b046-4423-90c5-0767eb8e4e4b None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:48.871137 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3033fb-b046-4423-90c5-0767eb8e4e4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:48.871924 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3033fb-b046-4423-90c5-0767eb8e4e4b None admin] RBAC: Adding query filter params (role.id=6a985d5659a3416cba33ca9d30b39f85, scope.project.id=e319c5f287b64c0d824e38968057b3b7, user.id=873ad929e6dd4a1c9f5875c6017d0fe0) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:28:48.872907 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3033fb-b046-4423-90c5-0767eb8e4e4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:48.874594 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3033fb-b046-4423-90c5-0767eb8e4e4b None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:48.881235 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 51/97] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:28:48 2022] GET /identity/v3/role_assignments?user.id=873ad929e6dd4a1c9f5875c6017d0fe0&scope.project.id=e319c5f287b64c0d824e38968057b3b7&role.id=6a985d5659a3416cba33ca9d30b39f85 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:50.159988 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee6722-7dba-4791-a6ea-9c5c64a70d91 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:50.160494 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee6722-7dba-4791-a6ea-9c5c64a70d91 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:50.160566 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee6722-7dba-4791-a6ea-9c5c64a70d91 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:50.162041 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 47/98] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:28:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:28:50.167075 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da4123-9c11-447b-874b-0813985b6774 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:50.167514 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da4123-9c11-447b-874b-0813985b6774 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:50.167675 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da4123-9c11-447b-874b-0813985b6774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:50.194757 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-d4da4123-9c11-447b-874b-0813985b6774 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:28:50.196003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-d4da4123-9c11-447b-874b-0813985b6774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:28:50.272788 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 52/99] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:28:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:28:50.693887 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b6e0f-4808-4adf-b84b-7726657b896b None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:50.694389 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b6e0f-4808-4adf-b84b-7726657b896b None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:50.694783 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b6e0f-4808-4adf-b84b-7726657b896b None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:50.696265 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 48/100] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:28:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:28:50.712764 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2707ba-1843-455c-87de-9de9f7382b25 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:50.713047 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2707ba-1843-455c-87de-9de9f7382b25 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:50.713331 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2707ba-1843-455c-87de-9de9f7382b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:50.721276 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac544c68-0f94-4ecb-867e-f1c53ad5ec45 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:50.721559 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac544c68-0f94-4ecb-867e-f1c53ad5ec45 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:50.721835 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac544c68-0f94-4ecb-867e-f1c53ad5ec45 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:50.723171 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 49/101] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:28:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:28:50.728281 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb91c7-35f6-4942-81e8-d9b597303cf6 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:50.728568 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb91c7-35f6-4942-81e8-d9b597303cf6 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:50.728844 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb91c7-35f6-4942-81e8-d9b597303cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:50.757484 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-7e2707ba-1843-455c-87de-9de9f7382b25 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:28:50.758745 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-7e2707ba-1843-455c-87de-9de9f7382b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:28:50.770303 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-23bb91c7-35f6-4942-81e8-d9b597303cf6 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:28:50.771618 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-23bb91c7-35f6-4942-81e8-d9b597303cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:28:50.864414 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:50.893309 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 50/102] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:28:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:28:50.894490 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bb91c7-35f6-4942-81e8-d9b597303cf6 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:50.901374 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 53/103] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:28:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:28:50.946745 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:51.023066 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bb91c7-35f6-4942-81e8-d9b597303cf6 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:51.062652 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503f6a3b-bf2c-4c23-a4b2-198d31815b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:51.063674 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f6a3b-bf2c-4c23-a4b2-198d31815b5a None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:51.064207 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f6a3b-bf2c-4c23-a4b2-198d31815b5a None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:51.064671 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f6a3b-bf2c-4c23-a4b2-198d31815b5a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:51.076010 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503f6a3b-bf2c-4c23-a4b2-198d31815b5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:28:51.085556 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-503f6a3b-bf2c-4c23-a4b2-198d31815b5a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:28:51.085556 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:28:51.085556 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:28:51.085556 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:28:51.085556 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:28:51.085556 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:28:51.085556 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:28:51.085556 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:28:51.085556 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:28:51.085556 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:28:51.085556 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:28:51.085556 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:28:51.085556 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:28:51.085556 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:28:51.085556 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:28:51.085556 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:28:51.086431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:28:51.086431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:28:51.086431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:28:51.086431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:28:51.086431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:28:51.086431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:28:51.086431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:28:51.086431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:28:51.086431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:28:51.086431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:28:51.086431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:28:51.086431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:28:51.086431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:28:51.086431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:28:51.086431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:28:51.086431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:28:51.087182 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:28:51.087182 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:28:51.087182 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:28:51.087182 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:28:51.087182 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:28:51.087182 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:28:51.087182 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:28:51.087182 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:28:51.087182 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:28:51.087182 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:28:51.087182 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:28:51.087182 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:28:51.089743 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 51/104] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:28:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:28:51.096001 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2707ba-1843-455c-87de-9de9f7382b25 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:51.183767 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936215e8-195c-4abf-af99-c614bbd8b3d9 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:51.185256 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936215e8-195c-4abf-af99-c614bbd8b3d9 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:51.185724 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936215e8-195c-4abf-af99-c614bbd8b3d9 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:51.187846 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 52/105] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:28:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:28:51.194600 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1facb-e14a-41e6-aae3-84e61f5e71ef None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:51.195109 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1facb-e14a-41e6-aae3-84e61f5e71ef None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:51.195628 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1facb-e14a-41e6-aae3-84e61f5e71ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:51.238083 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-a9f1facb-e14a-41e6-aae3-84e61f5e71ef None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:28:51.239734 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-a9f1facb-e14a-41e6-aae3-84e61f5e71ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:28:51.287339 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2707ba-1843-455c-87de-9de9f7382b25 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:51.351187 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd99c02a-ac19-4c02-a1e4-b45bc49e8267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:51.351972 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 53/106] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:28:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:28:51.352080 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99c02a-ac19-4c02-a1e4-b45bc49e8267 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:51.352080 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99c02a-ac19-4c02-a1e4-b45bc49e8267 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:51.352353 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99c02a-ac19-4c02-a1e4-b45bc49e8267 None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:51.352651 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f1facb-e14a-41e6-aae3-84e61f5e71ef None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:51.353027 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd99c02a-ac19-4c02-a1e4-b45bc49e8267 None admin] RBAC: Adding query filter params (name=member) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:28:51.353333 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd99c02a-ac19-4c02-a1e4-b45bc49e8267 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:51.354850 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd99c02a-ac19-4c02-a1e4-b45bc49e8267 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:51.374429 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 54/107] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 30 14:28:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:51.375308 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd99c02a-ac19-4c02-a1e4-b45bc49e8267 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:51.395596 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:51.449631 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:51.511578 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd99c02a-ac19-4c02-a1e4-b45bc49e8267 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:51.542098 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f1facb-e14a-41e6-aae3-84e61f5e71ef None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:51.548203 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fbeffe-b49c-4deb-b4ab-53dae1e08653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:51.548843 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbeffe-b49c-4deb-b4ab-53dae1e08653 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:51.549129 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbeffe-b49c-4deb-b4ab-53dae1e08653 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:51.549538 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbeffe-b49c-4deb-b4ab-53dae1e08653 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:51.560830 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fbeffe-b49c-4deb-b4ab-53dae1e08653 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:28:51.566337 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-01fbeffe-b49c-4deb-b4ab-53dae1e08653 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:28:51.566337 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:28:51.566337 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:28:51.566337 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:28:51.566337 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:28:51.566337 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:28:51.566337 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:28:51.566337 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:28:51.566337 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:28:51.566337 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:28:51.566337 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:28:51.566337 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:28:51.566337 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:28:51.566337 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:28:51.566337 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:28:51.566337 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:28:51.567161 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:28:51.567161 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:28:51.567161 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:28:51.567161 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:28:51.567161 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:28:51.567161 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:28:51.567161 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:28:51.567161 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:28:51.567161 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:28:51.567161 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:28:51.567161 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:28:51.567161 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:28:51.567161 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:28:51.567161 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:28:51.567161 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:28:51.567161 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:28:51.567956 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:28:51.567956 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:28:51.567956 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:28:51.567956 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:28:51.567956 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:28:51.567956 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:28:51.567956 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:28:51.567956 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:28:51.567956 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:28:51.567956 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:28:51.567956 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:28:51.567956 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:28:51.567956 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 55/108] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 30 14:28:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:28:51.568618 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fbeffe-b49c-4deb-b4ab-53dae1e08653 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:51.616878 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2a4ccb-05f2-4eaa-9513-e9256ffcb89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:51.617533 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a4ccb-05f2-4eaa-9513-e9256ffcb89c None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:51.617817 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a4ccb-05f2-4eaa-9513-e9256ffcb89c None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:51.618082 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a4ccb-05f2-4eaa-9513-e9256ffcb89c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:51.623915 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2a4ccb-05f2-4eaa-9513-e9256ffcb89c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:51.625418 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2a4ccb-05f2-4eaa-9513-e9256ffcb89c None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:51.627087 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 54/109] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:28:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:51.627984 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2a4ccb-05f2-4eaa-9513-e9256ffcb89c None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:51.678819 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:51.682788 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fbeffe-b49c-4deb-b4ab-53dae1e08653 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:51.727927 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc681b1-f00e-4a22-a9bf-b55906057976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:51.728603 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc681b1-f00e-4a22-a9bf-b55906057976 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:51.728973 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc681b1-f00e-4a22-a9bf-b55906057976 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:51.729322 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc681b1-f00e-4a22-a9bf-b55906057976 None admin] PATH_INFO: `/v3/users/873ad929e6dd4a1c9f5875c6017d0fe0` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:51.730635 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc681b1-f00e-4a22-a9bf-b55906057976 None admin] RBAC: Authorizing `identity:get_user(user_id=873ad929e6dd4a1c9f5875c6017d0fe0)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:51.733004 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc681b1-f00e-4a22-a9bf-b55906057976 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:51.735036 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 56/110] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:28:51 2022] GET /identity/v3/users/873ad929e6dd4a1c9f5875c6017d0fe0 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:51.735858 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc681b1-f00e-4a22-a9bf-b55906057976 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:51.798849 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:51.803635 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2a4ccb-05f2-4eaa-9513-e9256ffcb89c None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:51.838881 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8a05cb-bb38-4e69-8f75-9d8cbbb8584a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:51.839529 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a05cb-bb38-4e69-8f75-9d8cbbb8584a None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:51.839811 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a05cb-bb38-4e69-8f75-9d8cbbb8584a None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:51.840078 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a05cb-bb38-4e69-8f75-9d8cbbb8584a None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:51.840764 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8a05cb-bb38-4e69-8f75-9d8cbbb8584a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:28:51.841594 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8a05cb-bb38-4e69-8f75-9d8cbbb8584a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:51.842969 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8a05cb-bb38-4e69-8f75-9d8cbbb8584a None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:51.847167 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc681b1-f00e-4a22-a9bf-b55906057976 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:51.850864 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 55/111] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:28:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:51.855518 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782285e-4e78-4986-a063-86c91d84d9f1 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:51.855518 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782285e-4e78-4986-a063-86c91d84d9f1 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:51.855518 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782285e-4e78-4986-a063-86c91d84d9f1 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:51.855518 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 56/112] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:28:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:28:51.855518 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6782285e-4e78-4986-a063-86c91d84d9f1 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:51.885733 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415cd5d9-2af5-44c4-86fa-ee2fcca7b6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:51.886341 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415cd5d9-2af5-44c4-86fa-ee2fcca7b6d5 None admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:51.886626 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415cd5d9-2af5-44c4-86fa-ee2fcca7b6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:51.886907 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415cd5d9-2af5-44c4-86fa-ee2fcca7b6d5 None admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:51.887629 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415cd5d9-2af5-44c4-86fa-ee2fcca7b6d5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:51.889596 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415cd5d9-2af5-44c4-86fa-ee2fcca7b6d5 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:51.915644 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 57/113] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Aug 30 14:28:51 2022] POST /identity/v3/projects => generated 302 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:28:51.916466 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415cd5d9-2af5-44c4-86fa-ee2fcca7b6d5 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:51.940706 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:51.969170 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6782285e-4e78-4986-a063-86c91d84d9f1 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:52.008398 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65e6d26-3812-4f01-bb00-a21536756a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:52.009048 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e6d26-3812-4f01-bb00-a21536756a58 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:52.009497 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e6d26-3812-4f01-bb00-a21536756a58 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:52.009783 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e6d26-3812-4f01-bb00-a21536756a58 None admin] PATH_INFO: `/v3/projects/1c2139beef6b44b9a0574047298c1af9` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:52.011692 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65e6d26-3812-4f01-bb00-a21536756a58 None admin] RBAC: Authorizing `identity:get_project(project_id=1c2139beef6b44b9a0574047298c1af9)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:52.013931 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65e6d26-3812-4f01-bb00-a21536756a58 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:52.020796 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 57/114] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:28:51 2022] GET /identity/v3/projects/1c2139beef6b44b9a0574047298c1af9 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:52.020796 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34248df9-785b-440a-b095-ed669b3c376d None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:52.020796 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34248df9-785b-440a-b095-ed669b3c376d None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:52.020796 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34248df9-785b-440a-b095-ed669b3c376d None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:52.020796 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 58/115] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:28:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:28:52.021931 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34248df9-785b-440a-b095-ed669b3c376d None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:52.030432 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415cd5d9-2af5-44c4-86fa-ee2fcca7b6d5 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:52.058956 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:52.071600 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdda8d4-24d8-4da5-8ac3-02086bd90f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:52.072248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdda8d4-24d8-4da5-8ac3-02086bd90f56 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:52.072537 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdda8d4-24d8-4da5-8ac3-02086bd90f56 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:52.072814 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdda8d4-24d8-4da5-8ac3-02086bd90f56 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:52.083518 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdda8d4-24d8-4da5-8ac3-02086bd90f56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:28:52.088378 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-0fdda8d4-24d8-4da5-8ac3-02086bd90f56 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:28:52.088378 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:28:52.088378 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:28:52.088378 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:28:52.088378 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:28:52.088378 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:28:52.088378 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:28:52.088378 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:28:52.088378 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:28:52.088378 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:28:52.088378 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:28:52.088378 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:28:52.088378 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:28:52.088378 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:28:52.088378 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:28:52.088378 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:28:52.089248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:28:52.089248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:28:52.089248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:28:52.089248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:28:52.089248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:28:52.089248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:28:52.089248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:28:52.089248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:28:52.089248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:28:52.089248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:28:52.089248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:28:52.089248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:28:52.089248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:28:52.089248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:28:52.089248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:28:52.089248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:28:52.090009 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:28:52.090009 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:28:52.090009 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:28:52.090009 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:28:52.090009 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:28:52.090009 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:28:52.090009 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:28:52.090009 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:28:52.090009 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:28:52.090009 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:28:52.090009 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:28:52.090009 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:28:52.090009 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 58/116] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:28:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:28:52.091789 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df44814-65fa-44e2-85e3-79f4654d9c37 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:52.092077 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df44814-65fa-44e2-85e3-79f4654d9c37 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:52.092642 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df44814-65fa-44e2-85e3-79f4654d9c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:52.110518 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-2df44814-65fa-44e2-85e3-79f4654d9c37 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:28:52.111759 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-2df44814-65fa-44e2-85e3-79f4654d9c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:28:52.149569 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34248df9-785b-440a-b095-ed669b3c376d None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:52.185399 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 59/117] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:28:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:28:52.185982 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df44814-65fa-44e2-85e3-79f4654d9c37 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:52.192894 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093eeedb-6ac6-4ed1-bad8-3464e41b499f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:52.194128 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093eeedb-6ac6-4ed1-bad8-3464e41b499f None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:52.194649 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093eeedb-6ac6-4ed1-bad8-3464e41b499f None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:52.195701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093eeedb-6ac6-4ed1-bad8-3464e41b499f None admin] PATH_INFO: `/v3/users/4a989d2c0a4a49eb9303e4db6cc0b7e8` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:52.197103 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093eeedb-6ac6-4ed1-bad8-3464e41b499f None admin] RBAC: Authorizing `identity:get_user(user_id=4a989d2c0a4a49eb9303e4db6cc0b7e8)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:52.199427 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:52.201279 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093eeedb-6ac6-4ed1-bad8-3464e41b499f None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:52.203558 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 59/118] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:28:52 2022] GET /identity/v3/users/4a989d2c0a4a49eb9303e4db6cc0b7e8 => generated 247 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:52.205031 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae328f1-181e-473f-b8a3-c63da65964b3 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:52.205513 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae328f1-181e-473f-b8a3-c63da65964b3 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:52.206009 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae328f1-181e-473f-b8a3-c63da65964b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:52.231235 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-5ae328f1-181e-473f-b8a3-c63da65964b3 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:28:52.233041 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-5ae328f1-181e-473f-b8a3-c63da65964b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:28:52.314943 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df44814-65fa-44e2-85e3-79f4654d9c37 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:52.325231 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 60/119] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:28:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:28:52.326763 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae328f1-181e-473f-b8a3-c63da65964b3 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:52.360217 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:52.384280 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73ae48c-038a-487c-9b1d-56fee28f53e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:52.384672 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ae48c-038a-487c-9b1d-56fee28f53e1 None admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:52.384949 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ae48c-038a-487c-9b1d-56fee28f53e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:52.385219 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ae48c-038a-487c-9b1d-56fee28f53e1 None admin] PATH_INFO: `/v3/projects/1c2139beef6b44b9a0574047298c1af9/users/873ad929e6dd4a1c9f5875c6017d0fe0/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:52.392902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73ae48c-038a-487c-9b1d-56fee28f53e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=1c2139beef6b44b9a0574047298c1af9, user_id=873ad929e6dd4a1c9f5875c6017d0fe0, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:52.394427 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73ae48c-038a-487c-9b1d-56fee28f53e1 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:52.419953 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 60/120] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 30 14:28:52 2022] PUT /identity/v3/projects/1c2139beef6b44b9a0574047298c1af9/users/873ad929e6dd4a1c9f5875c6017d0fe0/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:28:52.421689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73ae48c-038a-487c-9b1d-56fee28f53e1 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:52.490244 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae328f1-181e-473f-b8a3-c63da65964b3 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:52.549373 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:52.560730 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a40983-a94c-45bb-a6cd-903dd93f5cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:52.561409 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a40983-a94c-45bb-a6cd-903dd93f5cb5 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:52.561685 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a40983-a94c-45bb-a6cd-903dd93f5cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:52.561951 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a40983-a94c-45bb-a6cd-903dd93f5cb5 None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:52.562676 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a40983-a94c-45bb-a6cd-903dd93f5cb5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:28:52.562979 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a40983-a94c-45bb-a6cd-903dd93f5cb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:52.566747 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a40983-a94c-45bb-a6cd-903dd93f5cb5 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:52.580256 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 61/121] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 30 14:28:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:52.581496 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a40983-a94c-45bb-a6cd-903dd93f5cb5 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:52.606930 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73ae48c-038a-487c-9b1d-56fee28f53e1 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:52.646221 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0a409c-d6d3-4ccd-9ed7-2d7c6d980d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:52.646856 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0a409c-d6d3-4ccd-9ed7-2d7c6d980d8b None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:52.647138 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0a409c-d6d3-4ccd-9ed7-2d7c6d980d8b None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:52.647440 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0a409c-d6d3-4ccd-9ed7-2d7c6d980d8b None admin] PATH_INFO: `/v3/projects/1c2139beef6b44b9a0574047298c1af9` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:52.648574 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0a409c-d6d3-4ccd-9ed7-2d7c6d980d8b None admin] RBAC: Authorizing `identity:get_project(project_id=1c2139beef6b44b9a0574047298c1af9)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:52.650090 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0a409c-d6d3-4ccd-9ed7-2d7c6d980d8b None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:52.652207 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 61/122] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:28:52 2022] GET /identity/v3/projects/1c2139beef6b44b9a0574047298c1af9 => generated 298 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:52.652988 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0a409c-d6d3-4ccd-9ed7-2d7c6d980d8b None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:52.699638 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:52.710128 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a40983-a94c-45bb-a6cd-903dd93f5cb5 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:52.755520 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef305aa7-8275-490f-949e-1affe4b3e1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:52.756183 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef305aa7-8275-490f-949e-1affe4b3e1dc None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:52.756470 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef305aa7-8275-490f-949e-1affe4b3e1dc None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:52.756751 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef305aa7-8275-490f-949e-1affe4b3e1dc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:52.771680 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef305aa7-8275-490f-949e-1affe4b3e1dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:28:52.779768 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-ef305aa7-8275-490f-949e-1affe4b3e1dc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:28:52.779768 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:28:52.779768 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:28:52.779768 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:28:52.779768 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:28:52.779768 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:28:52.779768 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:28:52.779768 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:28:52.779768 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:28:52.779768 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:28:52.779768 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:28:52.779768 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:28:52.779768 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:28:52.779768 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:28:52.779768 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:28:52.779768 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:28:52.780603 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:28:52.780603 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:28:52.780603 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:28:52.780603 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:28:52.780603 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:28:52.780603 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:28:52.780603 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:28:52.780603 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:28:52.780603 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:28:52.780603 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:28:52.780603 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:28:52.780603 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:28:52.780603 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:28:52.780603 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:28:52.780603 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:28:52.780603 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:28:52.781367 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:28:52.781367 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:28:52.781367 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:28:52.781367 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:28:52.781367 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:28:52.781367 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:28:52.781367 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:28:52.781367 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:28:52.781367 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:28:52.781367 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:28:52.781367 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:28:52.781367 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:28:52.784040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 62/123] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:28:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:28:52.785009 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef305aa7-8275-490f-949e-1affe4b3e1dc None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:52.786101 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0a409c-d6d3-4ccd-9ed7-2d7c6d980d8b None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:52.829457 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27528927-67c0-4cad-af2a-66cb3566c09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:52.830491 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27528927-67c0-4cad-af2a-66cb3566c09c None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:52.830964 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27528927-67c0-4cad-af2a-66cb3566c09c None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:52.831464 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27528927-67c0-4cad-af2a-66cb3566c09c None admin] PATH_INFO: `/v3/users/d6285aeb36494642a235b3b6c197315a` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:52.841148 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:52.852065 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27528927-67c0-4cad-af2a-66cb3566c09c None admin] RBAC: Authorizing `identity:get_user(user_id=d6285aeb36494642a235b3b6c197315a)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:52.853688 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27528927-67c0-4cad-af2a-66cb3566c09c None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:52.856037 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 62/124] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:28:52 2022] GET /identity/v3/users/d6285aeb36494642a235b3b6c197315a => generated 289 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:52.857491 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27528927-67c0-4cad-af2a-66cb3566c09c None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:52.918629 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef305aa7-8275-490f-949e-1affe4b3e1dc None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:52.956856 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf6ebf5-ff47-45c5-8f95-c9023043e57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:52.957544 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf6ebf5-ff47-45c5-8f95-c9023043e57b None admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:52.957887 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf6ebf5-ff47-45c5-8f95-c9023043e57b None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:52.958171 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf6ebf5-ff47-45c5-8f95-c9023043e57b None admin] PATH_INFO: `/v3/projects/1c2139beef6b44b9a0574047298c1af9/users/4a989d2c0a4a49eb9303e4db6cc0b7e8/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:52.960303 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf6ebf5-ff47-45c5-8f95-c9023043e57b None admin] RBAC: Authorizing `identity:create_grant(project_id=1c2139beef6b44b9a0574047298c1af9, user_id=4a989d2c0a4a49eb9303e4db6cc0b7e8, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:52.962282 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf6ebf5-ff47-45c5-8f95-c9023043e57b None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:52.980566 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 63/125] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 30 14:28:52 2022] PUT /identity/v3/projects/1c2139beef6b44b9a0574047298c1af9/users/4a989d2c0a4a49eb9303e4db6cc0b7e8/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:28:52.981332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf6ebf5-ff47-45c5-8f95-c9023043e57b None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:52.993731 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27528927-67c0-4cad-af2a-66cb3566c09c None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:53.000056 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:53.041265 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb36e24d-5332-42e1-abec-c2c3ccd96bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:53.041923 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36e24d-5332-42e1-abec-c2c3ccd96bc6 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:53.042783 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36e24d-5332-42e1-abec-c2c3ccd96bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:53.043131 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36e24d-5332-42e1-abec-c2c3ccd96bc6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:53.059701 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb36e24d-5332-42e1-abec-c2c3ccd96bc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:28:53.067351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-fb36e24d-5332-42e1-abec-c2c3ccd96bc6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:28:53.067351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:28:53.067351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:28:53.067351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:28:53.067351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:28:53.067351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:28:53.067351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:28:53.067351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:28:53.067351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:28:53.067351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:28:53.067351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:28:53.067351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:28:53.067351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:28:53.067351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:28:53.067351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:28:53.067351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:28:53.068275 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:28:53.068275 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:28:53.068275 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:28:53.068275 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:28:53.068275 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:28:53.068275 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:28:53.068275 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:28:53.068275 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:28:53.068275 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:28:53.068275 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:28:53.068275 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:28:53.068275 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:28:53.068275 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:28:53.068275 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:28:53.068275 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:28:53.068275 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:28:53.069066 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:28:53.069066 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:28:53.069066 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:28:53.069066 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:28:53.069066 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:28:53.069066 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:28:53.069066 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:28:53.069066 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:28:53.069066 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:28:53.069066 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:28:53.069066 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:28:53.069066 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:28:53.069066 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 63/126] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 30 14:28:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:28:53.069741 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb36e24d-5332-42e1-abec-c2c3ccd96bc6 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:53.121843 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf6ebf5-ff47-45c5-8f95-c9023043e57b None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:53.141212 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:53.166994 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcdf332-998f-46e6-839e-5510a7f9aa4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:53.167816 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcdf332-998f-46e6-839e-5510a7f9aa4c None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:53.168208 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcdf332-998f-46e6-839e-5510a7f9aa4c None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:53.168585 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcdf332-998f-46e6-839e-5510a7f9aa4c None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:53.169912 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcdf332-998f-46e6-839e-5510a7f9aa4c None admin] RBAC: Adding query filter params (name=member) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:28:53.170320 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcdf332-998f-46e6-839e-5510a7f9aa4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:53.171989 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcdf332-998f-46e6-839e-5510a7f9aa4c None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:53.181827 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 64/127] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 30 14:28:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:53.182906 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcdf332-998f-46e6-839e-5510a7f9aa4c None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:53.239871 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb36e24d-5332-42e1-abec-c2c3ccd96bc6 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:53.265534 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:53.276484 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb70fb0-0287-45a3-8d37-56d9d6fc9a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:53.277411 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb70fb0-0287-45a3-8d37-56d9d6fc9a24 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:53.277865 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb70fb0-0287-45a3-8d37-56d9d6fc9a24 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:53.278394 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb70fb0-0287-45a3-8d37-56d9d6fc9a24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:53.304003 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcdf332-998f-46e6-839e-5510a7f9aa4c None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:53.343679 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c9952-7187-41ea-bf7a-e1def302d09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:53.343871 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c9952-7187-41ea-bf7a-e1def302d09c None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:53.344173 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c9952-7187-41ea-bf7a-e1def302d09c None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:53.344521 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c9952-7187-41ea-bf7a-e1def302d09c None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:53.345320 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402c9952-7187-41ea-bf7a-e1def302d09c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:28:53.345791 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402c9952-7187-41ea-bf7a-e1def302d09c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:53.347465 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402c9952-7187-41ea-bf7a-e1def302d09c None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:53.353358 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb70fb0-0287-45a3-8d37-56d9d6fc9a24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:53.355397 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb70fb0-0287-45a3-8d37-56d9d6fc9a24 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:53.360311 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 65/128] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:28:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:53.360311 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c9952-7187-41ea-bf7a-e1def302d09c None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:53.423308 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:53.476073 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c9952-7187-41ea-bf7a-e1def302d09c None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:53.497010 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 64/129] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:28:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:28:53.498769 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb70fb0-0287-45a3-8d37-56d9d6fc9a24 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:53.548338 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeef60d-c809-477c-ad5d-9a3f35363237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:53.550797 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeef60d-c809-477c-ad5d-9a3f35363237 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:53.556423 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeef60d-c809-477c-ad5d-9a3f35363237 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:53.556929 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeef60d-c809-477c-ad5d-9a3f35363237 None admin] PATH_INFO: `/v3/users/873ad929e6dd4a1c9f5875c6017d0fe0` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:53.558677 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeef60d-c809-477c-ad5d-9a3f35363237 None admin] RBAC: Authorizing `identity:get_user(user_id=873ad929e6dd4a1c9f5875c6017d0fe0)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:53.560912 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeef60d-c809-477c-ad5d-9a3f35363237 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:53.562734 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 66/130] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:28:53 2022] GET /identity/v3/users/873ad929e6dd4a1c9f5875c6017d0fe0 => generated 275 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:53.566139 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeef60d-c809-477c-ad5d-9a3f35363237 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:53.571031 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:53.676462 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb70fb0-0287-45a3-8d37-56d9d6fc9a24 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:53.738778 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f5568-815f-439a-96bc-f7ec7d7fd317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:53.743240 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f5568-815f-439a-96bc-f7ec7d7fd317 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:53.746236 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f5568-815f-439a-96bc-f7ec7d7fd317 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:53.746350 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f5568-815f-439a-96bc-f7ec7d7fd317 None admin] PATH_INFO: `/v3/users/873ad929e6dd4a1c9f5875c6017d0fe0` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:53.749130 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1f5568-815f-439a-96bc-f7ec7d7fd317 None admin] RBAC: Authorizing `identity:get_user(user_id=873ad929e6dd4a1c9f5875c6017d0fe0)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:53.750604 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1f5568-815f-439a-96bc-f7ec7d7fd317 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:53.755312 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 65/131] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:28:53 2022] GET /identity/v3/users/873ad929e6dd4a1c9f5875c6017d0fe0 => generated 275 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:53.763605 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f5568-815f-439a-96bc-f7ec7d7fd317 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:53.774237 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeef60d-c809-477c-ad5d-9a3f35363237 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:53.799709 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:53.911006 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f5568-815f-439a-96bc-f7ec7d7fd317 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:53.956356 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ba3d17-6671-44c5-99be-2e5c576c1b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:53.957411 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba3d17-6671-44c5-99be-2e5c576c1b90 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:53.957891 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba3d17-6671-44c5-99be-2e5c576c1b90 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:53.961475 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba3d17-6671-44c5-99be-2e5c576c1b90 None admin] PATH_INFO: `/v3/projects/1c2139beef6b44b9a0574047298c1af9` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:53.964211 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ba3d17-6671-44c5-99be-2e5c576c1b90 None admin] RBAC: Authorizing `identity:get_project(project_id=1c2139beef6b44b9a0574047298c1af9)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:53.966516 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ba3d17-6671-44c5-99be-2e5c576c1b90 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:53.969411 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 67/132] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:28:53 2022] GET /identity/v3/projects/1c2139beef6b44b9a0574047298c1af9 => generated 298 bytes in 404 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:53.975049 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ba3d17-6671-44c5-99be-2e5c576c1b90 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:54.013479 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d59bda7-62a3-4c9b-9ae5-948f1e31bdf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:54.014150 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59bda7-62a3-4c9b-9ae5-948f1e31bdf6 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:54.014434 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59bda7-62a3-4c9b-9ae5-948f1e31bdf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:54.014708 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59bda7-62a3-4c9b-9ae5-948f1e31bdf6 None admin] PATH_INFO: `/v3/projects/e319c5f287b64c0d824e38968057b3b7` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:54.015875 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d59bda7-62a3-4c9b-9ae5-948f1e31bdf6 None admin] RBAC: Authorizing `identity:get_project(project_id=e319c5f287b64c0d824e38968057b3b7)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:54.017675 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d59bda7-62a3-4c9b-9ae5-948f1e31bdf6 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:54.031235 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 66/133] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:28:53 2022] GET /identity/v3/projects/e319c5f287b64c0d824e38968057b3b7 => generated 312 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:54.031235 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d59bda7-62a3-4c9b-9ae5-948f1e31bdf6 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:54.102260 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:54.149046 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ba3d17-6671-44c5-99be-2e5c576c1b90 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:54.207954 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3466046a-001b-4f58-a4d8-228b50c54f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:54.208527 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3466046a-001b-4f58-a4d8-228b50c54f04 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:54.208813 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3466046a-001b-4f58-a4d8-228b50c54f04 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:54.209091 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3466046a-001b-4f58-a4d8-228b50c54f04 None admin] PATH_INFO: `/v3/projects/1c2139beef6b44b9a0574047298c1af9` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:54.213053 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3466046a-001b-4f58-a4d8-228b50c54f04 None admin] RBAC: Authorizing `identity:get_project(project_id=1c2139beef6b44b9a0574047298c1af9)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:54.214668 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3466046a-001b-4f58-a4d8-228b50c54f04 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:54.217865 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 68/134] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:28:53 2022] GET /identity/v3/projects/1c2139beef6b44b9a0574047298c1af9 => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:54.219459 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3466046a-001b-4f58-a4d8-228b50c54f04 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:54.283116 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:54.296740 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d59bda7-62a3-4c9b-9ae5-948f1e31bdf6 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:54.340258 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3466046a-001b-4f58-a4d8-228b50c54f04 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:54.351628 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5590cda-e078-43c0-a39d-533bdeffb04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:54.352520 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5590cda-e078-43c0-a39d-533bdeffb04d None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:54.353448 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5590cda-e078-43c0-a39d-533bdeffb04d None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:54.354524 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5590cda-e078-43c0-a39d-533bdeffb04d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:54.355469 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5590cda-e078-43c0-a39d-533bdeffb04d None admin] RBAC: Adding query filter params (role.id=f51dfe78039041e3a947ea27cfe97651, scope.project.id=1c2139beef6b44b9a0574047298c1af9, user.id=d6285aeb36494642a235b3b6c197315a) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:28:54.356191 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5590cda-e078-43c0-a39d-533bdeffb04d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:54.359162 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5590cda-e078-43c0-a39d-533bdeffb04d None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:54.368818 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 67/135] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:28:54 2022] GET /identity/v3/role_assignments?user.id=d6285aeb36494642a235b3b6c197315a&scope.project.id=1c2139beef6b44b9a0574047298c1af9&role.id=f51dfe78039041e3a947ea27cfe97651 => generated 259 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:54.371122 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c25d42-13e6-414a-af36-7636ef210609 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:54.372163 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c25d42-13e6-414a-af36-7636ef210609 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:54.372747 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c25d42-13e6-414a-af36-7636ef210609 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:54.375010 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 68/136] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:28:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:28:54.376015 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c25d42-13e6-414a-af36-7636ef210609 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:54.386394 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b721aaaf-5171-4eb0-97f9-ef112d67c4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:54.387284 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b721aaaf-5171-4eb0-97f9-ef112d67c4ec None admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:54.387783 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b721aaaf-5171-4eb0-97f9-ef112d67c4ec None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:54.388232 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b721aaaf-5171-4eb0-97f9-ef112d67c4ec None admin] PATH_INFO: `/v3/projects/e319c5f287b64c0d824e38968057b3b7/users/873ad929e6dd4a1c9f5875c6017d0fe0/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:54.391022 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b721aaaf-5171-4eb0-97f9-ef112d67c4ec None admin] RBAC: Authorizing `identity:create_grant(project_id=e319c5f287b64c0d824e38968057b3b7, user_id=873ad929e6dd4a1c9f5875c6017d0fe0, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:54.392538 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b721aaaf-5171-4eb0-97f9-ef112d67c4ec None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:54.408100 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:54.416421 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 69/137] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 30 14:28:54 2022] PUT /identity/v3/projects/e319c5f287b64c0d824e38968057b3b7/users/873ad929e6dd4a1c9f5875c6017d0fe0/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:28:54.417782 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f4a712-3731-4d81-b4a7-f395444309fc None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:54.418084 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f4a712-3731-4d81-b4a7-f395444309fc None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:54.422945 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f4a712-3731-4d81-b4a7-f395444309fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:54.484692 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-93f4a712-3731-4d81-b4a7-f395444309fc None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:28:54.485914 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-93f4a712-3731-4d81-b4a7-f395444309fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:28:54.573677 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c25d42-13e6-414a-af36-7636ef210609 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:54.654711 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 70/138] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:28:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:28:54.666788 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd50c4e-bbcb-41f9-b529-0a8f3bc5568d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:54.667519 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd50c4e-bbcb-41f9-b529-0a8f3bc5568d None admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:54.667792 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd50c4e-bbcb-41f9-b529-0a8f3bc5568d None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:54.668071 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd50c4e-bbcb-41f9-b529-0a8f3bc5568d None admin] PATH_INFO: `/v3/projects/1c2139beef6b44b9a0574047298c1af9/users/873ad929e6dd4a1c9f5875c6017d0fe0/roles/1c4886a7a7fd402d8221a0158ddcb167` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:54.670490 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd50c4e-bbcb-41f9-b529-0a8f3bc5568d None admin] RBAC: Authorizing `identity:create_grant(project_id=1c2139beef6b44b9a0574047298c1af9, user_id=873ad929e6dd4a1c9f5875c6017d0fe0, role_id=1c4886a7a7fd402d8221a0158ddcb167)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:54.672036 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd50c4e-bbcb-41f9-b529-0a8f3bc5568d None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:54.689003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 69/139] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 30 14:28:54 2022] PUT /identity/v3/projects/1c2139beef6b44b9a0574047298c1af9/users/873ad929e6dd4a1c9f5875c6017d0fe0/roles/1c4886a7a7fd402d8221a0158ddcb167 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:28:54.977699 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864872fc-e2db-4994-a21b-26e562aaf3a5 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:54.978070 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864872fc-e2db-4994-a21b-26e562aaf3a5 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:54.978259 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864872fc-e2db-4994-a21b-26e562aaf3a5 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:54.979786 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 71/140] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:28:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:28:54.986107 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b1161-9f9b-4ae0-8e52-ab095d913cfd None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:54.986392 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b1161-9f9b-4ae0-8e52-ab095d913cfd None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:54.986669 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b1161-9f9b-4ae0-8e52-ab095d913cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:55.008538 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-388b1161-9f9b-4ae0-8e52-ab095d913cfd None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:28:55.010170 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-388b1161-9f9b-4ae0-8e52-ab095d913cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:28:55.024198 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:55.091562 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 70/141] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:28:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:28:55.272890 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864872fc-e2db-4994-a21b-26e562aaf3a5 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:55.396468 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864872fc-e2db-4994-a21b-26e562aaf3a5 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:55.434177 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:55.443867 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e92c8c-43f3-4a64-ab14-ba75d5d350f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:55.444512 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e92c8c-43f3-4a64-ab14-ba75d5d350f3 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:55.444793 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e92c8c-43f3-4a64-ab14-ba75d5d350f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:55.445061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e92c8c-43f3-4a64-ab14-ba75d5d350f3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:55.446193 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e92c8c-43f3-4a64-ab14-ba75d5d350f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:55.447656 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e92c8c-43f3-4a64-ab14-ba75d5d350f3 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:55.450235 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 72/142] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:28:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:55.455034 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b1161-9f9b-4ae0-8e52-ab095d913cfd None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:55.491060 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e92c8c-43f3-4a64-ab14-ba75d5d350f3 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:55.575940 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b1161-9f9b-4ae0-8e52-ab095d913cfd None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:55.622057 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90110335-5b81-4be9-8af0-ac787aecad4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:55.622681 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90110335-5b81-4be9-8af0-ac787aecad4c None admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:55.622966 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90110335-5b81-4be9-8af0-ac787aecad4c None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:55.623272 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90110335-5b81-4be9-8af0-ac787aecad4c None admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:55.623940 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90110335-5b81-4be9-8af0-ac787aecad4c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:55.625407 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90110335-5b81-4be9-8af0-ac787aecad4c None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:55.664690 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 71/143] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 30 14:28:55 2022] POST /identity/v3/users => generated 283 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:28:55.707082 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b5ff7-5ad3-4d7e-a919-b6a2d80c3c50 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:55.707438 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b5ff7-5ad3-4d7e-a919-b6a2d80c3c50 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:55.707867 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b5ff7-5ad3-4d7e-a919-b6a2d80c3c50 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:55.709913 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 72/144] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:28:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:28:55.715292 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c083b-40f2-4d36-8bdf-17b6bd7498ce None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:55.716854 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c083b-40f2-4d36-8bdf-17b6bd7498ce None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:55.717145 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c083b-40f2-4d36-8bdf-17b6bd7498ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:55.727186 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:55.751150 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e92c8c-43f3-4a64-ab14-ba75d5d350f3 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:55.754683 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-f32c083b-40f2-4d36-8bdf-17b6bd7498ce None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:28:55.755980 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-f32c083b-40f2-4d36-8bdf-17b6bd7498ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:28:55.836138 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da64381-f2bb-42a6-804a-1242dd35f0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:55.838198 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da64381-f2bb-42a6-804a-1242dd35f0e0 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:55.839458 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da64381-f2bb-42a6-804a-1242dd35f0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:55.839942 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da64381-f2bb-42a6-804a-1242dd35f0e0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:55.862492 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da64381-f2bb-42a6-804a-1242dd35f0e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:28:55.867791 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-7da64381-f2bb-42a6-804a-1242dd35f0e0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:28:55.867791 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:28:55.867791 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:28:55.867791 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:28:55.867791 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:28:55.867791 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:28:55.867791 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:28:55.867791 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:28:55.867791 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:28:55.867791 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:28:55.867791 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:28:55.867791 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:28:55.867791 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:28:55.867791 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:28:55.867791 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:28:55.867791 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:28:55.868624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:28:55.868624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:28:55.868624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:28:55.868624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:28:55.868624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:28:55.868624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:28:55.868624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:28:55.868624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:28:55.868624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:28:55.868624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:28:55.868624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:28:55.868624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:28:55.868624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:28:55.868624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:28:55.868624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:28:55.868624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:28:55.869381 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:28:55.869381 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:28:55.869381 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:28:55.869381 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:28:55.869381 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:28:55.869381 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:28:55.869381 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:28:55.869381 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:28:55.869381 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:28:55.869381 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:28:55.869381 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:28:55.869381 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:28:55.871758 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 73/145] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:28:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 381 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:28:55.878583 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da64381-f2bb-42a6-804a-1242dd35f0e0 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:55.903062 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 73/146] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:28:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:28:55.963124 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:56.020094 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da64381-f2bb-42a6-804a-1242dd35f0e0 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:56.066444 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33b1eda-b881-44d2-bd1c-a2960e14b0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:56.067500 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b1eda-b881-44d2-bd1c-a2960e14b0f4 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:56.067997 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b1eda-b881-44d2-bd1c-a2960e14b0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:56.068468 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b1eda-b881-44d2-bd1c-a2960e14b0f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:56.069371 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33b1eda-b881-44d2-bd1c-a2960e14b0f4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:28:56.069874 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33b1eda-b881-44d2-bd1c-a2960e14b0f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:56.071566 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33b1eda-b881-44d2-bd1c-a2960e14b0f4 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:56.084203 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 74/147] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 30 14:28:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:56.089499 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32c083b-40f2-4d36-8bdf-17b6bd7498ce None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:56.213075 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32c083b-40f2-4d36-8bdf-17b6bd7498ce None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:56.253125 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:56.259360 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc911b3-3c02-4d74-a8c9-327a2b9f89b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:56.260357 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc911b3-3c02-4d74-a8c9-327a2b9f89b9 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:56.260833 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc911b3-3c02-4d74-a8c9-327a2b9f89b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:56.261302 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc911b3-3c02-4d74-a8c9-327a2b9f89b9 None admin] PATH_INFO: `/v3/users/873ad929e6dd4a1c9f5875c6017d0fe0` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:56.263272 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc911b3-3c02-4d74-a8c9-327a2b9f89b9 None admin] RBAC: Authorizing `identity:get_user(user_id=873ad929e6dd4a1c9f5875c6017d0fe0)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:56.265634 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc911b3-3c02-4d74-a8c9-327a2b9f89b9 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:56.267677 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 74/148] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:28:56 2022] GET /identity/v3/users/873ad929e6dd4a1c9f5875c6017d0fe0 => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:56.275088 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33b1eda-b881-44d2-bd1c-a2960e14b0f4 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:56.357591 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc911b3-3c02-4d74-a8c9-327a2b9f89b9 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:56.401818 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33b1eda-b881-44d2-bd1c-a2960e14b0f4 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:56.424832 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:56.449311 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ac71e-0be2-48ee-b699-49777a5e4a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:56.451482 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ac71e-0be2-48ee-b699-49777a5e4a62 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:56.451618 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ac71e-0be2-48ee-b699-49777a5e4a62 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:56.451803 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ac71e-0be2-48ee-b699-49777a5e4a62 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:56.498912 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc911b3-3c02-4d74-a8c9-327a2b9f89b9 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:56.541534 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66ac71e-0be2-48ee-b699-49777a5e4a62 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:56.543684 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66ac71e-0be2-48ee-b699-49777a5e4a62 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:56.544156 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ae0345-d0be-4fe3-afbf-1dd3278852bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:56.545098 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae0345-d0be-4fe3-afbf-1dd3278852bb None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:56.545573 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae0345-d0be-4fe3-afbf-1dd3278852bb None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:56.546143 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae0345-d0be-4fe3-afbf-1dd3278852bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:56.565986 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ae0345-d0be-4fe3-afbf-1dd3278852bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:28:56.572860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-b3ae0345-d0be-4fe3-afbf-1dd3278852bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:28:56.572860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:28:56.572860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:28:56.572860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:28:56.572860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:28:56.572860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:28:56.572860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:28:56.572860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:28:56.572860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:28:56.572860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:28:56.572860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:28:56.572860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:28:56.572860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:28:56.572860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:28:56.572860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:28:56.572860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:28:56.573912 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:28:56.573912 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:28:56.573912 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:28:56.573912 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:28:56.573912 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:28:56.573912 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:28:56.573912 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:28:56.573912 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:28:56.573912 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:28:56.573912 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:28:56.573912 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:28:56.573912 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:28:56.573912 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:28:56.573912 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:28:56.573912 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:28:56.573912 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:28:56.574865 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:28:56.574865 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:28:56.574865 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:28:56.574865 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:28:56.574865 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:28:56.574865 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:28:56.574865 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:28:56.574865 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:28:56.574865 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:28:56.574865 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:28:56.574865 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:28:56.574865 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:28:56.577157 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 75/149] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 30 14:28:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:28:56.584451 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ae0345-d0be-4fe3-afbf-1dd3278852bb None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:56.666706 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 75/150] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:28:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:28:56.674309 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ac71e-0be2-48ee-b699-49777a5e4a62 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:56.700844 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:56.712368 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:56.788306 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ae0345-d0be-4fe3-afbf-1dd3278852bb None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:56.800713 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ac71e-0be2-48ee-b699-49777a5e4a62 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:56.840231 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e814750d-a0a2-4f16-9864-58a17a48667f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:56.841468 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814750d-a0a2-4f16-9864-58a17a48667f None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:56.841953 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814750d-a0a2-4f16-9864-58a17a48667f None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:56.843002 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814750d-a0a2-4f16-9864-58a17a48667f None admin] PATH_INFO: `/v3/projects/1c2139beef6b44b9a0574047298c1af9` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:56.844575 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e814750d-a0a2-4f16-9864-58a17a48667f None admin] RBAC: Authorizing `identity:get_project(project_id=1c2139beef6b44b9a0574047298c1af9)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:56.846234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e814750d-a0a2-4f16-9864-58a17a48667f None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:56.848280 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 76/151] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:28:56 2022] GET /identity/v3/projects/1c2139beef6b44b9a0574047298c1af9 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:28:56.855000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e814750d-a0a2-4f16-9864-58a17a48667f None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:56.915972 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6249ce7c-1fd3-44fe-9a58-2bd1af7fdfe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:56.916587 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249ce7c-1fd3-44fe-9a58-2bd1af7fdfe7 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:56.916917 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249ce7c-1fd3-44fe-9a58-2bd1af7fdfe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:56.917136 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249ce7c-1fd3-44fe-9a58-2bd1af7fdfe7 None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:56.917816 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6249ce7c-1fd3-44fe-9a58-2bd1af7fdfe7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:28:56.918117 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6249ce7c-1fd3-44fe-9a58-2bd1af7fdfe7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:56.919609 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6249ce7c-1fd3-44fe-9a58-2bd1af7fdfe7 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:56.949145 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 76/152] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:28:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:28:56.956275 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6249ce7c-1fd3-44fe-9a58-2bd1af7fdfe7 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:56.963586 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:56.984858 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e814750d-a0a2-4f16-9864-58a17a48667f None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:57.037527 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37999436-3e98-42a9-9744-047c4431efd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:57.038171 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37999436-3e98-42a9-9744-047c4431efd4 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:57.038459 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37999436-3e98-42a9-9744-047c4431efd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:57.038735 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37999436-3e98-42a9-9744-047c4431efd4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:57.039732 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37999436-3e98-42a9-9744-047c4431efd4 None admin] RBAC: Adding query filter params (role.id=6a985d5659a3416cba33ca9d30b39f85, scope.project.id=1c2139beef6b44b9a0574047298c1af9, user.id=873ad929e6dd4a1c9f5875c6017d0fe0) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:28:57.040741 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37999436-3e98-42a9-9744-047c4431efd4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:57.042377 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37999436-3e98-42a9-9744-047c4431efd4 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:57.049233 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 77/153] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:28:56 2022] GET /identity/v3/role_assignments?user.id=873ad929e6dd4a1c9f5875c6017d0fe0&scope.project.id=1c2139beef6b44b9a0574047298c1af9&role.id=6a985d5659a3416cba33ca9d30b39f85 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:28:57.117472 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afaeec5-163d-49cc-9aed-e74c55d848a1 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:57.118281 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afaeec5-163d-49cc-9aed-e74c55d848a1 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:57.119181 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afaeec5-163d-49cc-9aed-e74c55d848a1 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:57.121306 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 78/154] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:28:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:28:57.126969 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc19b7b0-7e84-4d97-9796-d174b09adc50 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:57.127543 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc19b7b0-7e84-4d97-9796-d174b09adc50 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:57.127997 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc19b7b0-7e84-4d97-9796-d174b09adc50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:57.158853 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-bc19b7b0-7e84-4d97-9796-d174b09adc50 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:28:57.160981 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-bc19b7b0-7e84-4d97-9796-d174b09adc50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:28:57.233308 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6249ce7c-1fd3-44fe-9a58-2bd1af7fdfe7 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:57.294305 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:57.305601 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 79/155] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:28:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:28:57.320754 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed87bbb-0e3e-4819-a243-5976f7a3dfb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:57.321393 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed87bbb-0e3e-4819-a243-5976f7a3dfb8 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:57.321671 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed87bbb-0e3e-4819-a243-5976f7a3dfb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:57.321943 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed87bbb-0e3e-4819-a243-5976f7a3dfb8 None admin] PATH_INFO: `/v3/users/4a989d2c0a4a49eb9303e4db6cc0b7e8` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:57.327975 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed87bbb-0e3e-4819-a243-5976f7a3dfb8 None admin] RBAC: Authorizing `identity:get_user(user_id=4a989d2c0a4a49eb9303e4db6cc0b7e8)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:57.329456 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed87bbb-0e3e-4819-a243-5976f7a3dfb8 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:57.331112 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 77/156] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:28:56 2022] GET /identity/v3/users/4a989d2c0a4a49eb9303e4db6cc0b7e8 => generated 247 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:57.336887 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc19b7b0-7e84-4d97-9796-d174b09adc50 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:57.451242 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:57.474810 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc19b7b0-7e84-4d97-9796-d174b09adc50 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:57.514640 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41699b3-bd61-45fd-93d5-613f6ac60d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:57.515273 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41699b3-bd61-45fd-93d5-613f6ac60d45 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:57.515552 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41699b3-bd61-45fd-93d5-613f6ac60d45 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:57.515820 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41699b3-bd61-45fd-93d5-613f6ac60d45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:57.598697 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41699b3-bd61-45fd-93d5-613f6ac60d45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:57.600219 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41699b3-bd61-45fd-93d5-613f6ac60d45 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:57.699477 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:57.726720 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 80/157] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:28:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:28:57.732853 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed87bbb-0e3e-4819-a243-5976f7a3dfb8 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:57.884894 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed87bbb-0e3e-4819-a243-5976f7a3dfb8 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:57.930308 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615420d5-4290-47a5-b0ec-e228906782dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:57.931383 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615420d5-4290-47a5-b0ec-e228906782dc None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:57.931913 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615420d5-4290-47a5-b0ec-e228906782dc None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:57.932370 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615420d5-4290-47a5-b0ec-e228906782dc None admin] PATH_INFO: `/v3/projects/1c2139beef6b44b9a0574047298c1af9` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:57.934018 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615420d5-4290-47a5-b0ec-e228906782dc None admin] RBAC: Authorizing `identity:get_project(project_id=1c2139beef6b44b9a0574047298c1af9)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:57.936241 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615420d5-4290-47a5-b0ec-e228906782dc None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:57.939116 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 78/158] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:28:57 2022] GET /identity/v3/projects/1c2139beef6b44b9a0574047298c1af9 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:57.943932 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41699b3-bd61-45fd-93d5-613f6ac60d45 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:57.967563 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615420d5-4290-47a5-b0ec-e228906782dc None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:58.047858 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:58.118770 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615420d5-4290-47a5-b0ec-e228906782dc None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:58.126936 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41699b3-bd61-45fd-93d5-613f6ac60d45 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:58.164990 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abdbafd-23e7-4964-a7fd-fc5f35079f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:58.165869 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abdbafd-23e7-4964-a7fd-fc5f35079f0e None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:58.166291 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abdbafd-23e7-4964-a7fd-fc5f35079f0e None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:58.166701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abdbafd-23e7-4964-a7fd-fc5f35079f0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:58.167596 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abdbafd-23e7-4964-a7fd-fc5f35079f0e None admin] RBAC: Adding query filter params (role.id=3cda0615d87d4af0860a2a5e9b4e2c05, scope.project.id=1c2139beef6b44b9a0574047298c1af9, user.id=4a989d2c0a4a49eb9303e4db6cc0b7e8) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:28:58.168043 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abdbafd-23e7-4964-a7fd-fc5f35079f0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:58.170348 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abdbafd-23e7-4964-a7fd-fc5f35079f0e None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:58.176560 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 81/159] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:28:57 2022] GET /identity/v3/role_assignments?user.id=4a989d2c0a4a49eb9303e4db6cc0b7e8&scope.project.id=1c2139beef6b44b9a0574047298c1af9&role.id=3cda0615d87d4af0860a2a5e9b4e2c05 => generated 608 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:58.185735 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11d78df-58f7-4a38-a175-9e3c24d72051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:58.186394 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11d78df-58f7-4a38-a175-9e3c24d72051 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:58.186671 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11d78df-58f7-4a38-a175-9e3c24d72051 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:58.186940 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11d78df-58f7-4a38-a175-9e3c24d72051 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:58.215280 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11d78df-58f7-4a38-a175-9e3c24d72051 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:28:58.217017 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-e11d78df-58f7-4a38-a175-9e3c24d72051 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:28:58.217017 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:28:58.217017 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:28:58.217017 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:28:58.217017 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:28:58.217017 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:28:58.217017 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:28:58.217017 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:28:58.217017 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:28:58.217017 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:28:58.217017 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:28:58.217017 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:28:58.217017 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:28:58.217017 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:28:58.217017 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:28:58.217017 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:28:58.218031 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:28:58.218031 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:28:58.218031 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:28:58.218031 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:28:58.218031 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:28:58.218031 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:28:58.218031 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:28:58.218031 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:28:58.218031 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:28:58.218031 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:28:58.218031 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:28:58.218031 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:28:58.218031 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:28:58.218031 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:28:58.218031 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:28:58.218031 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:28:58.218791 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:28:58.218791 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:28:58.218791 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:28:58.218791 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:28:58.218791 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:28:58.218791 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:28:58.218791 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:28:58.218791 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:28:58.218791 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:28:58.218791 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:28:58.218791 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:28:58.218791 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:28:58.222876 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 79/160] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:28:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:28:58.226108 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abdbafd-23e7-4964-a7fd-fc5f35079f0e None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:58.294882 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:58.334751 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee78f9fe-3601-4951-bf46-af76986a63f9 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:58.335087 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee78f9fe-3601-4951-bf46-af76986a63f9 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:58.335334 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee78f9fe-3601-4951-bf46-af76986a63f9 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:58.339438 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 80/161] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:28:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:28:58.345077 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db46022-ee57-41a6-905f-cd28becf6ebf None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:58.345377 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db46022-ee57-41a6-905f-cd28becf6ebf None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:58.345625 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db46022-ee57-41a6-905f-cd28becf6ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:58.380601 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-0db46022-ee57-41a6-905f-cd28becf6ebf None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:28:58.381924 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-0db46022-ee57-41a6-905f-cd28becf6ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:28:58.394007 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abdbafd-23e7-4964-a7fd-fc5f35079f0e None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:58.449672 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f351741-95c8-4f93-ab92-6bed1eb286cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:58.450970 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f351741-95c8-4f93-ab92-6bed1eb286cf None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:58.451499 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f351741-95c8-4f93-ab92-6bed1eb286cf None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:58.452406 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f351741-95c8-4f93-ab92-6bed1eb286cf None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:58.453338 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f351741-95c8-4f93-ab92-6bed1eb286cf None admin] RBAC: Adding query filter params (name=reader) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:28:58.453846 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f351741-95c8-4f93-ab92-6bed1eb286cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:58.455371 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f351741-95c8-4f93-ab92-6bed1eb286cf None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:58.464735 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 82/162] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 30 14:28:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:58.466402 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a20595-0768-4a97-9b2c-00ef10d09696 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:58.466915 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a20595-0768-4a97-9b2c-00ef10d09696 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:58.467408 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a20595-0768-4a97-9b2c-00ef10d09696 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:58.469155 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 83/163] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:28:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:28:58.475763 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b4505-6b8c-4aba-9c47-442297c363a9 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:58.476295 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b4505-6b8c-4aba-9c47-442297c363a9 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:58.476775 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b4505-6b8c-4aba-9c47-442297c363a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:58.492699 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 81/164] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:28:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:28:58.493916 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db46022-ee57-41a6-905f-cd28becf6ebf None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:58.496402 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-4b9b4505-6b8c-4aba-9c47-442297c363a9 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:28:58.498527 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-4b9b4505-6b8c-4aba-9c47-442297c363a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:28:58.568957 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:58.570581 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 84/165] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:28:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:28:58.645713 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db46022-ee57-41a6-905f-cd28becf6ebf None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:58.703548 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689de4af-bfea-404d-af4a-9011bb933fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:58.704193 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689de4af-bfea-404d-af4a-9011bb933fde None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:58.704472 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689de4af-bfea-404d-af4a-9011bb933fde None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:58.704746 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689de4af-bfea-404d-af4a-9011bb933fde None admin] PATH_INFO: `/v3/users/d6285aeb36494642a235b3b6c197315a` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:58.705909 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689de4af-bfea-404d-af4a-9011bb933fde None admin] RBAC: Authorizing `identity:get_user(user_id=d6285aeb36494642a235b3b6c197315a)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:58.707389 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689de4af-bfea-404d-af4a-9011bb933fde None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:58.708983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 82/166] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:28:58 2022] GET /identity/v3/users/d6285aeb36494642a235b3b6c197315a => generated 289 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:58.714496 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9b4505-6b8c-4aba-9c47-442297c363a9 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:58.752041 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:58.831870 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9b4505-6b8c-4aba-9c47-442297c363a9 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:58.877364 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc822ec-0372-45e7-9fab-1ba8a95be22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:58.878000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc822ec-0372-45e7-9fab-1ba8a95be22a None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:58.878283 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc822ec-0372-45e7-9fab-1ba8a95be22a None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:58.878553 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc822ec-0372-45e7-9fab-1ba8a95be22a None admin] PATH_INFO: `/v3/projects/1c2139beef6b44b9a0574047298c1af9` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:58.883877 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc822ec-0372-45e7-9fab-1ba8a95be22a None admin] RBAC: Authorizing `identity:get_project(project_id=1c2139beef6b44b9a0574047298c1af9)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:58.885321 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc822ec-0372-45e7-9fab-1ba8a95be22a None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:58.887266 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 85/167] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:28:58 2022] GET /identity/v3/projects/1c2139beef6b44b9a0574047298c1af9 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:58.892864 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689de4af-bfea-404d-af4a-9011bb933fde None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:58.969270 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc822ec-0372-45e7-9fab-1ba8a95be22a None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:59.030265 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689de4af-bfea-404d-af4a-9011bb933fde None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:59.060616 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:59.104844 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba5c2d-6861-4f5b-8958-c9a2712bea89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:59.105765 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba5c2d-6861-4f5b-8958-c9a2712bea89 None admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:59.106213 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba5c2d-6861-4f5b-8958-c9a2712bea89 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:59.106728 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba5c2d-6861-4f5b-8958-c9a2712bea89 None admin] PATH_INFO: `/v3/projects/1c2139beef6b44b9a0574047298c1af9/users/d6285aeb36494642a235b3b6c197315a/roles/f51dfe78039041e3a947ea27cfe97651` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:59.112516 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ba5c2d-6861-4f5b-8958-c9a2712bea89 None admin] RBAC: Authorizing `identity:create_grant(project_id=1c2139beef6b44b9a0574047298c1af9, user_id=d6285aeb36494642a235b3b6c197315a, role_id=f51dfe78039041e3a947ea27cfe97651)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:59.114158 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ba5c2d-6861-4f5b-8958-c9a2712bea89 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:59.141528 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 83/168] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 30 14:28:58 2022] PUT /identity/v3/projects/1c2139beef6b44b9a0574047298c1af9/users/d6285aeb36494642a235b3b6c197315a/roles/f51dfe78039041e3a947ea27cfe97651 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:28:59.187635 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba5c2d-6861-4f5b-8958-c9a2712bea89 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:59.188308 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc822ec-0372-45e7-9fab-1ba8a95be22a None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:59.190993 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:59.271735 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fdd230-75c6-43c0-aaf8-cd35ddc9bf80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:59.272335 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdd230-75c6-43c0-aaf8-cd35ddc9bf80 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:59.272602 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdd230-75c6-43c0-aaf8-cd35ddc9bf80 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:59.272880 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdd230-75c6-43c0-aaf8-cd35ddc9bf80 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:59.299482 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fdd230-75c6-43c0-aaf8-cd35ddc9bf80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:28:59.304580 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-66fdd230-75c6-43c0-aaf8-cd35ddc9bf80 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:28:59.304580 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:28:59.304580 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:28:59.304580 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:28:59.304580 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:28:59.304580 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:28:59.304580 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:28:59.304580 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:28:59.304580 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:28:59.304580 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:28:59.304580 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:28:59.304580 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:28:59.304580 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:28:59.304580 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:28:59.304580 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:28:59.304580 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:28:59.305471 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:28:59.305471 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:28:59.305471 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:28:59.305471 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:28:59.305471 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:28:59.305471 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:28:59.305471 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:28:59.305471 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:28:59.305471 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:28:59.305471 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:28:59.305471 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:28:59.305471 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:28:59.305471 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:28:59.305471 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:28:59.305471 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:28:59.305471 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:28:59.306350 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:28:59.306350 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:28:59.306350 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:28:59.306350 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:28:59.306350 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:28:59.306350 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:28:59.306350 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:28:59.306350 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:28:59.306350 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:28:59.306350 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:28:59.306350 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:28:59.306350 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:28:59.306944 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 86/169] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:28:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:28:59.314555 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fdd230-75c6-43c0-aaf8-cd35ddc9bf80 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:59.421300 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba5c2d-6861-4f5b-8958-c9a2712bea89 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:59.457223 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fdd230-75c6-43c0-aaf8-cd35ddc9bf80 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:59.483895 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff904b5-a77f-46d1-8cba-dbefe700a470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:59.484538 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff904b5-a77f-46d1-8cba-dbefe700a470 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:59.484817 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff904b5-a77f-46d1-8cba-dbefe700a470 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:59.485087 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff904b5-a77f-46d1-8cba-dbefe700a470 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:59.504118 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff904b5-a77f-46d1-8cba-dbefe700a470 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:28:59.509801 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-4ff904b5-a77f-46d1-8cba-dbefe700a470 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:28:59.509801 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:28:59.509801 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:28:59.509801 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:28:59.509801 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:28:59.509801 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:28:59.509801 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:28:59.509801 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:28:59.509801 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:28:59.509801 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:28:59.509801 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:28:59.509801 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:28:59.509801 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:28:59.509801 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:28:59.509801 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:28:59.509801 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:28:59.510729 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:28:59.510729 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:28:59.510729 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:28:59.510729 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:28:59.510729 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:28:59.510729 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:28:59.510729 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:28:59.510729 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:28:59.510729 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:28:59.510729 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:28:59.510729 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:28:59.510729 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:28:59.510729 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:28:59.510729 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:28:59.510729 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:28:59.510729 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:28:59.511585 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:28:59.511585 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:28:59.511585 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:28:59.511585 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:28:59.511585 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:28:59.511585 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:28:59.511585 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:28:59.511585 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:28:59.511585 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:28:59.511585 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:28:59.511585 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:28:59.511585 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:28:59.512214 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 84/170] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 30 14:28:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:28:59.522858 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a164c8-d98b-4d6b-a638-1d27931c7e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:59.523437 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a164c8-d98b-4d6b-a638-1d27931c7e94 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:59.523714 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a164c8-d98b-4d6b-a638-1d27931c7e94 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:59.523982 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a164c8-d98b-4d6b-a638-1d27931c7e94 None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:59.524652 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a164c8-d98b-4d6b-a638-1d27931c7e94 None admin] RBAC: Adding query filter params (name=member) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:28:59.524952 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a164c8-d98b-4d6b-a638-1d27931c7e94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:59.526380 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a164c8-d98b-4d6b-a638-1d27931c7e94 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:59.530472 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff904b5-a77f-46d1-8cba-dbefe700a470 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:59.548992 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 87/171] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 30 14:28:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:59.557366 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a164c8-d98b-4d6b-a638-1d27931c7e94 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:59.595700 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:59.617009 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:28:59.763484 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff904b5-a77f-46d1-8cba-dbefe700a470 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:59.796209 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a164c8-d98b-4d6b-a638-1d27931c7e94 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:28:59.849708 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb5fa80-1415-419d-a6b1-c2e0e0e9e36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:59.850337 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5fa80-1415-419d-a6b1-c2e0e0e9e36a None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:59.850619 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5fa80-1415-419d-a6b1-c2e0e0e9e36a None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:59.850891 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5fa80-1415-419d-a6b1-c2e0e0e9e36a None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:59.851598 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb5fa80-1415-419d-a6b1-c2e0e0e9e36a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:28:59.851900 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb5fa80-1415-419d-a6b1-c2e0e0e9e36a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:59.853775 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb5fa80-1415-419d-a6b1-c2e0e0e9e36a None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:59.870486 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 85/172] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:28:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:59.878769 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec8514d-0e64-465b-9f2d-b9c7306627c3 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:59.879127 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec8514d-0e64-465b-9f2d-b9c7306627c3 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:59.879441 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec8514d-0e64-465b-9f2d-b9c7306627c3 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:59.880768 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 86/173] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:28:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:28:59.881824 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8d8891-91c0-4d4d-ac0e-f70618e1cbcb None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:59.882165 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8d8891-91c0-4d4d-ac0e-f70618e1cbcb None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:59.882442 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8d8891-91c0-4d4d-ac0e-f70618e1cbcb None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:59.883677 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 87/174] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:28:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:28:59.884424 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8d8891-91c0-4d4d-ac0e-f70618e1cbcb None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:28:59.886625 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b2eff-e1d9-4c69-b71b-d61fe4dcec88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:28:59.887452 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b2eff-e1d9-4c69-b71b-d61fe4dcec88 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:59.887741 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b2eff-e1d9-4c69-b71b-d61fe4dcec88 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:59.888083 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b2eff-e1d9-4c69-b71b-d61fe4dcec88 None admin] PATH_INFO: `/v3/users/873ad929e6dd4a1c9f5875c6017d0fe0` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:59.889506 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b2eff-e1d9-4c69-b71b-d61fe4dcec88 None admin] RBAC: Authorizing `identity:get_user(user_id=873ad929e6dd4a1c9f5875c6017d0fe0)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:28:59.890965 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b2eff-e1d9-4c69-b71b-d61fe4dcec88 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:28:59.892596 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 88/175] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:28:59 2022] GET /identity/v3/users/873ad929e6dd4a1c9f5875c6017d0fe0 => generated 275 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:28:59.893840 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c887b25-ebe8-436c-aaa4-03a89c71d445 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:28:59.894121 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c887b25-ebe8-436c-aaa4-03a89c71d445 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:28:59.894398 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c887b25-ebe8-436c-aaa4-03a89c71d445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:28:59.914048 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-9c887b25-ebe8-436c-aaa4-03a89c71d445 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:28:59.915247 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-9c887b25-ebe8-436c-aaa4-03a89c71d445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:00.003120 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:00.010117 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 89/176] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:28:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:00.010459 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8d8891-91c0-4d4d-ac0e-f70618e1cbcb None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:00.011411 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5534d1-2a5d-4a88-93c4-217d52dac77d None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:00.011697 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5534d1-2a5d-4a88-93c4-217d52dac77d None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:00.011971 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5534d1-2a5d-4a88-93c4-217d52dac77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:00.048072 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-ff5534d1-2a5d-4a88-93c4-217d52dac77d None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:00.052376 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-ff5534d1-2a5d-4a88-93c4-217d52dac77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:00.078805 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a7952b-f3bb-40e7-b1ae-7db531ebb5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:00.080505 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a7952b-f3bb-40e7-b1ae-7db531ebb5c8 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:00.081160 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a7952b-f3bb-40e7-b1ae-7db531ebb5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:00.082865 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a7952b-f3bb-40e7-b1ae-7db531ebb5c8 None admin] PATH_INFO: `/v3/users/873ad929e6dd4a1c9f5875c6017d0fe0` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:00.084406 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a7952b-f3bb-40e7-b1ae-7db531ebb5c8 None admin] RBAC: Authorizing `identity:get_user(user_id=873ad929e6dd4a1c9f5875c6017d0fe0)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:00.086953 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a7952b-f3bb-40e7-b1ae-7db531ebb5c8 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:00.092658 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 88/177] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:28:59 2022] GET /identity/v3/users/873ad929e6dd4a1c9f5875c6017d0fe0 => generated 275 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:00.093633 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a7952b-f3bb-40e7-b1ae-7db531ebb5c8 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:00.094103 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:00.188668 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 90/178] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:00.190535 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5534d1-2a5d-4a88-93c4-217d52dac77d None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:00.317555 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a7952b-f3bb-40e7-b1ae-7db531ebb5c8 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:00.392939 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdba440-ddfd-4f25-8e67-6d6a66cfbbbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:00.393571 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdba440-ddfd-4f25-8e67-6d6a66cfbbbc None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:00.393855 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdba440-ddfd-4f25-8e67-6d6a66cfbbbc None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:00.394128 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdba440-ddfd-4f25-8e67-6d6a66cfbbbc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:00.419410 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5534d1-2a5d-4a88-93c4-217d52dac77d None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:00.432418 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:00.467701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad5598-ba32-48f0-aafb-733864b043a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:00.468599 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad5598-ba32-48f0-aafb-733864b043a2 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:00.468882 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad5598-ba32-48f0-aafb-733864b043a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:00.469155 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad5598-ba32-48f0-aafb-733864b043a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:00.473933 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdba440-ddfd-4f25-8e67-6d6a66cfbbbc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:00.475431 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdba440-ddfd-4f25-8e67-6d6a66cfbbbc None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:00.503595 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:00.563112 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ad5598-ba32-48f0-aafb-733864b043a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:00.564627 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ad5598-ba32-48f0-aafb-733864b043a2 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:00.582618 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 89/179] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:29:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:29:00.583435 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdba440-ddfd-4f25-8e67-6d6a66cfbbbc None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:00.659462 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:00.679611 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 91/180] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:29:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:29:00.686157 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dbc302-3634-427d-b366-f528d560e320 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:00.686446 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dbc302-3634-427d-b366-f528d560e320 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:00.686717 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dbc302-3634-427d-b366-f528d560e320 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:00.688015 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 92/181] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:00.692592 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dbc302-3634-427d-b366-f528d560e320 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:00.698206 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdba440-ddfd-4f25-8e67-6d6a66cfbbbc None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:00.735374 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb6e225-76ca-4dff-a872-8087941efc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:00.736016 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb6e225-76ca-4dff-a872-8087941efc17 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:00.736297 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb6e225-76ca-4dff-a872-8087941efc17 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:00.736574 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb6e225-76ca-4dff-a872-8087941efc17 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:00.746708 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb6e225-76ca-4dff-a872-8087941efc17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:00.751895 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:00.752247 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-fdb6e225-76ca-4dff-a872-8087941efc17 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:29:00.752247 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:00.752247 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:00.752247 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:00.752247 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:00.752247 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:00.752247 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:00.752247 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:00.752247 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:00.752247 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:00.752247 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:00.752247 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:00.752247 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:00.752247 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:00.752247 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:00.752247 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:00.753110 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:00.753110 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:00.753110 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:00.753110 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:00.753110 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:00.753110 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:00.753110 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:00.753110 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:00.753110 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:00.753110 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:00.753110 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:00.753110 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:00.753110 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:00.753110 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:00.753110 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:00.753110 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:00.755983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:00.755983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:00.755983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:00.755983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:00.755983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:00.755983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:00.755983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:00.755983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:00.755983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:00.755983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:00.755983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:29:00.755983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:00.755983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 90/182] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 30 14:29:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:00.755983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb6e225-76ca-4dff-a872-8087941efc17 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:00.806793 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dbc302-3634-427d-b366-f528d560e320 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:00.846765 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796fbf56-b319-4570-919a-67ec85978008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:00.847447 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796fbf56-b319-4570-919a-67ec85978008 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:00.847726 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796fbf56-b319-4570-919a-67ec85978008 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:00.848277 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796fbf56-b319-4570-919a-67ec85978008 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:00.859568 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796fbf56-b319-4570-919a-67ec85978008 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:00.864511 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-796fbf56-b319-4570-919a-67ec85978008 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:29:00.864511 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:00.864511 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:00.864511 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:00.864511 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:00.864511 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:00.864511 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:00.864511 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:00.864511 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:00.864511 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:00.864511 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:00.864511 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:00.864511 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:00.864511 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:00.864511 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:00.864511 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:00.865373 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:00.865373 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:00.865373 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:00.865373 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:00.865373 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:00.865373 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:00.865373 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:00.865373 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:00.865373 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:00.865373 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:00.865373 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:00.865373 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:00.865373 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:00.865373 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:00.865373 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:00.865373 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:00.866169 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:00.866169 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:00.866169 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:00.866169 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:00.866169 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:00.866169 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:00.866169 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:00.866169 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:00.866169 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:00.866169 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:00.866169 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:29:00.866169 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:00.866169 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 93/183] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 30 14:29:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:00.866828 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796fbf56-b319-4570-919a-67ec85978008 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:00.873652 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb6e225-76ca-4dff-a872-8087941efc17 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:00.906208 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:00.915675 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f75677-689d-495b-9576-f437e9944845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:00.916422 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f75677-689d-495b-9576-f437e9944845 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:00.916710 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f75677-689d-495b-9576-f437e9944845 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:00.916988 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f75677-689d-495b-9576-f437e9944845 None admin] PATH_INFO: `/v3/projects/e319c5f287b64c0d824e38968057b3b7` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:00.918144 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f75677-689d-495b-9576-f437e9944845 None admin] RBAC: Authorizing `identity:get_project(project_id=e319c5f287b64c0d824e38968057b3b7)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:00.920357 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f75677-689d-495b-9576-f437e9944845 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:00.922018 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 91/184] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:29:00 2022] GET /identity/v3/projects/e319c5f287b64c0d824e38968057b3b7 => generated 312 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:00.926864 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627caa69-0b77-4285-aa84-d09cfd376a25 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:00.926864 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627caa69-0b77-4285-aa84-d09cfd376a25 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:00.926864 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627caa69-0b77-4285-aa84-d09cfd376a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:00.943438 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-627caa69-0b77-4285-aa84-d09cfd376a25 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:00.944627 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-627caa69-0b77-4285-aa84-d09cfd376a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:00.986863 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796fbf56-b319-4570-919a-67ec85978008 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:01.002021 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:01.016440 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 92/185] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:01.017320 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627caa69-0b77-4285-aa84-d09cfd376a25 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:01.031199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e138d2c3-de41-4633-bc0f-ee57f140fa35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:01.032493 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e138d2c3-de41-4633-bc0f-ee57f140fa35 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:01.032997 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e138d2c3-de41-4633-bc0f-ee57f140fa35 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:01.033762 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e138d2c3-de41-4633-bc0f-ee57f140fa35 None admin] PATH_INFO: `/v3/projects/1c2139beef6b44b9a0574047298c1af9` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:01.035959 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e138d2c3-de41-4633-bc0f-ee57f140fa35 None admin] RBAC: Authorizing `identity:get_project(project_id=1c2139beef6b44b9a0574047298c1af9)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:01.037894 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e138d2c3-de41-4633-bc0f-ee57f140fa35 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:01.040171 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 94/186] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:29:00 2022] GET /identity/v3/projects/1c2139beef6b44b9a0574047298c1af9 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:01.041214 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e138d2c3-de41-4633-bc0f-ee57f140fa35 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:01.132889 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627caa69-0b77-4285-aa84-d09cfd376a25 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:01.149088 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e138d2c3-de41-4633-bc0f-ee57f140fa35 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:01.160040 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:01.172885 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2197e7-2d4b-4c08-9ac1-b371ac82e89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:01.173492 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2197e7-2d4b-4c08-9ac1-b371ac82e89a None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:01.173764 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2197e7-2d4b-4c08-9ac1-b371ac82e89a None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:01.174041 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2197e7-2d4b-4c08-9ac1-b371ac82e89a None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:01.174709 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2197e7-2d4b-4c08-9ac1-b371ac82e89a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:01.175011 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2197e7-2d4b-4c08-9ac1-b371ac82e89a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:01.176498 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2197e7-2d4b-4c08-9ac1-b371ac82e89a None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:01.186035 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 93/187] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:29:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:01.186822 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2197e7-2d4b-4c08-9ac1-b371ac82e89a None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:01.191959 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e03a964-c49f-4542-b42f-d850c3d911df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:01.192838 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03a964-c49f-4542-b42f-d850c3d911df None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:01.193317 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03a964-c49f-4542-b42f-d850c3d911df None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:01.193802 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03a964-c49f-4542-b42f-d850c3d911df None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:01.194681 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e03a964-c49f-4542-b42f-d850c3d911df None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:01.195190 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e03a964-c49f-4542-b42f-d850c3d911df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:01.196941 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e03a964-c49f-4542-b42f-d850c3d911df None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:01.206800 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 95/188] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:29:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:01.213104 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e03a964-c49f-4542-b42f-d850c3d911df None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:01.265410 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:01.295817 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2197e7-2d4b-4c08-9ac1-b371ac82e89a None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:01.330766 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e03a964-c49f-4542-b42f-d850c3d911df None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:01.333252 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfd886f-28ce-4c8c-9e0e-dabbaa960b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:01.333937 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfd886f-28ce-4c8c-9e0e-dabbaa960b52 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:01.334225 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfd886f-28ce-4c8c-9e0e-dabbaa960b52 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:01.334542 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfd886f-28ce-4c8c-9e0e-dabbaa960b52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:01.335304 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfd886f-28ce-4c8c-9e0e-dabbaa960b52 None admin] RBAC: Adding query filter params (role.id=6a985d5659a3416cba33ca9d30b39f85, scope.project.id=e319c5f287b64c0d824e38968057b3b7, user.id=873ad929e6dd4a1c9f5875c6017d0fe0) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:01.335667 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfd886f-28ce-4c8c-9e0e-dabbaa960b52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:01.337114 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfd886f-28ce-4c8c-9e0e-dabbaa960b52 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:01.344384 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 94/189] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:29:01 2022] GET /identity/v3/role_assignments?user.id=873ad929e6dd4a1c9f5875c6017d0fe0&scope.project.id=e319c5f287b64c0d824e38968057b3b7&role.id=6a985d5659a3416cba33ca9d30b39f85 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:01.351401 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfd886f-28ce-4c8c-9e0e-dabbaa960b52 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:01.386847 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d9e96c-8c0a-4bfb-99e7-812f99a5dc06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:01.388298 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9e96c-8c0a-4bfb-99e7-812f99a5dc06 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:01.388588 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9e96c-8c0a-4bfb-99e7-812f99a5dc06 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:01.388862 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9e96c-8c0a-4bfb-99e7-812f99a5dc06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:01.389614 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d9e96c-8c0a-4bfb-99e7-812f99a5dc06 None admin] RBAC: Adding query filter params (role.id=1c4886a7a7fd402d8221a0158ddcb167, scope.project.id=1c2139beef6b44b9a0574047298c1af9, user.id=873ad929e6dd4a1c9f5875c6017d0fe0) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:01.389926 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d9e96c-8c0a-4bfb-99e7-812f99a5dc06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:01.392678 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d9e96c-8c0a-4bfb-99e7-812f99a5dc06 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:01.401346 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 96/190] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:29:01 2022] GET /identity/v3/role_assignments?user.id=873ad929e6dd4a1c9f5875c6017d0fe0&scope.project.id=1c2139beef6b44b9a0574047298c1af9&role.id=1c4886a7a7fd402d8221a0158ddcb167 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:01.402625 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d9e96c-8c0a-4bfb-99e7-812f99a5dc06 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:01.408901 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:01.488664 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfd886f-28ce-4c8c-9e0e-dabbaa960b52 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:01.542617 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d9e96c-8c0a-4bfb-99e7-812f99a5dc06 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:01.549686 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8752d9-49a0-47ea-afce-ece9d7556004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:01.550593 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8752d9-49a0-47ea-afce-ece9d7556004 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:01.551045 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8752d9-49a0-47ea-afce-ece9d7556004 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:01.551500 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8752d9-49a0-47ea-afce-ece9d7556004 None admin] PATH_INFO: `/v3/users/05d150e8f6f049bd906ecda0491014a6` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:01.556726 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:01.570069 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8752d9-49a0-47ea-afce-ece9d7556004 None admin] RBAC: Authorizing `identity:get_user(user_id=05d150e8f6f049bd906ecda0491014a6)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:01.571626 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8752d9-49a0-47ea-afce-ece9d7556004 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:01.573521 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 95/191] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:29:01 2022] GET /identity/v3/users/05d150e8f6f049bd906ecda0491014a6 => generated 283 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:01.574118 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8752d9-49a0-47ea-afce-ece9d7556004 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:01.598460 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a6876b-6efa-46d4-a72a-f41e420adcf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:01.599364 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a6876b-6efa-46d4-a72a-f41e420adcf1 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:01.599643 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a6876b-6efa-46d4-a72a-f41e420adcf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:01.599917 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a6876b-6efa-46d4-a72a-f41e420adcf1 None admin] PATH_INFO: `/v3/users/05d150e8f6f049bd906ecda0491014a6` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:01.601529 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a6876b-6efa-46d4-a72a-f41e420adcf1 None admin] RBAC: Authorizing `identity:get_user(user_id=05d150e8f6f049bd906ecda0491014a6)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:01.603006 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a6876b-6efa-46d4-a72a-f41e420adcf1 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:01.604828 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 97/192] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:29:01 2022] GET /identity/v3/users/05d150e8f6f049bd906ecda0491014a6 => generated 283 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:01.605855 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fdc633-7cad-426f-b992-62e2a42a6eb5 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:01.606232 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fdc633-7cad-426f-b992-62e2a42a6eb5 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:01.606483 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fdc633-7cad-426f-b992-62e2a42a6eb5 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:01.608584 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 98/193] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:01.609314 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fdc633-7cad-426f-b992-62e2a42a6eb5 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:01.707498 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8752d9-49a0-47ea-afce-ece9d7556004 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:01.717503 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:01.727549 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fdc633-7cad-426f-b992-62e2a42a6eb5 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:01.742987 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546bf4ee-d311-4d3e-a2a6-a19cb5799f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:01.743911 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546bf4ee-d311-4d3e-a2a6-a19cb5799f94 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:01.744184 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546bf4ee-d311-4d3e-a2a6-a19cb5799f94 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:01.744457 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546bf4ee-d311-4d3e-a2a6-a19cb5799f94 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:01.745565 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546bf4ee-d311-4d3e-a2a6-a19cb5799f94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:01.748062 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546bf4ee-d311-4d3e-a2a6-a19cb5799f94 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:01.749801 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 96/194] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:29:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:01.750387 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546bf4ee-d311-4d3e-a2a6-a19cb5799f94 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:01.766311 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8d5ddd-cda3-43a4-944f-46a2a133c51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:01.766916 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d5ddd-cda3-43a4-944f-46a2a133c51b None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:01.767193 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d5ddd-cda3-43a4-944f-46a2a133c51b None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:01.767493 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d5ddd-cda3-43a4-944f-46a2a133c51b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:01.816027 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:01.838848 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8d5ddd-cda3-43a4-944f-46a2a133c51b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:01.840578 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8d5ddd-cda3-43a4-944f-46a2a133c51b None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:01.849443 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546bf4ee-d311-4d3e-a2a6-a19cb5799f94 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:01.885321 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c07df-847f-4bdb-a101-9c77d85db557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:01.885929 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c07df-847f-4bdb-a101-9c77d85db557 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:01.886209 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c07df-847f-4bdb-a101-9c77d85db557 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:01.886480 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c07df-847f-4bdb-a101-9c77d85db557 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:01.930588 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:01.940566 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 99/195] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:29:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:29:01.940566 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe80ca62-4a3d-45ed-b927-63e1b62ae03d None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:01.940566 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe80ca62-4a3d-45ed-b927-63e1b62ae03d None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:01.940566 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe80ca62-4a3d-45ed-b927-63e1b62ae03d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:01.955654 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712c07df-847f-4bdb-a101-9c77d85db557 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:01.957121 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712c07df-847f-4bdb-a101-9c77d85db557 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:01.960309 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-fe80ca62-4a3d-45ed-b927-63e1b62ae03d None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:01.961440 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-fe80ca62-4a3d-45ed-b927-63e1b62ae03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:02.026465 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 100/196] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:02.035456 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe80ca62-4a3d-45ed-b927-63e1b62ae03d None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:02.035456 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:02.066910 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 97/197] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:29:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:29:02.066910 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c07df-847f-4bdb-a101-9c77d85db557 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:02.135840 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe80ca62-4a3d-45ed-b927-63e1b62ae03d None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:02.149505 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:02.174101 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a49eca-0d14-49d7-8419-49f6341a0f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:02.174715 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c07df-847f-4bdb-a101-9c77d85db557 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:02.174844 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a49eca-0d14-49d7-8419-49f6341a0f95 None admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:02.175001 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a49eca-0d14-49d7-8419-49f6341a0f95 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:02.175291 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a49eca-0d14-49d7-8419-49f6341a0f95 None admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:02.175943 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a49eca-0d14-49d7-8419-49f6341a0f95 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:02.177620 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a49eca-0d14-49d7-8419-49f6341a0f95 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:02.200476 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 101/198] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 30 14:29:02 2022] POST /identity/v3/users => generated 297 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:29:02.201170 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a49eca-0d14-49d7-8419-49f6341a0f95 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:02.230928 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73393e00-acd8-4b46-b28a-bdae32fa34cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:02.232271 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73393e00-acd8-4b46-b28a-bdae32fa34cf None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:02.232840 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73393e00-acd8-4b46-b28a-bdae32fa34cf None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:02.233377 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73393e00-acd8-4b46-b28a-bdae32fa34cf None admin] PATH_INFO: `/v3/projects/961f8057321f482d9a5d9202c7bc3c4c` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:02.235148 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73393e00-acd8-4b46-b28a-bdae32fa34cf None admin] RBAC: Authorizing `identity:get_project(project_id=961f8057321f482d9a5d9202c7bc3c4c)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:02.236995 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73393e00-acd8-4b46-b28a-bdae32fa34cf None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:02.239324 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 98/199] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:29:02 2022] GET /identity/v3/projects/961f8057321f482d9a5d9202c7bc3c4c => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:02.249065 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73393e00-acd8-4b46-b28a-bdae32fa34cf None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:02.273557 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:02.323863 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a49eca-0d14-49d7-8419-49f6341a0f95 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:02.363732 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3d7934-f7ea-41bc-bfec-c013d265b0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:02.364386 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3d7934-f7ea-41bc-bfec-c013d265b0c7 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:02.364663 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3d7934-f7ea-41bc-bfec-c013d265b0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:02.365450 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3d7934-f7ea-41bc-bfec-c013d265b0c7 None admin] PATH_INFO: `/v3/projects/961f8057321f482d9a5d9202c7bc3c4c` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:02.366999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3d7934-f7ea-41bc-bfec-c013d265b0c7 None admin] RBAC: Authorizing `identity:get_project(project_id=961f8057321f482d9a5d9202c7bc3c4c)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:02.368564 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3d7934-f7ea-41bc-bfec-c013d265b0c7 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:02.370294 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 102/200] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:29:02 2022] GET /identity/v3/projects/961f8057321f482d9a5d9202c7bc3c4c => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:02.371068 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3d7934-f7ea-41bc-bfec-c013d265b0c7 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:02.377164 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73393e00-acd8-4b46-b28a-bdae32fa34cf None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:02.416921 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd4452a-7824-40e2-9545-3557639af2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:02.417539 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd4452a-7824-40e2-9545-3557639af2ae None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:02.417817 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd4452a-7824-40e2-9545-3557639af2ae None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:02.418090 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd4452a-7824-40e2-9545-3557639af2ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:02.418954 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd4452a-7824-40e2-9545-3557639af2ae None admin] RBAC: Adding query filter params (role.id=3cda0615d87d4af0860a2a5e9b4e2c05, scope.project.id=961f8057321f482d9a5d9202c7bc3c4c, user.id=05d150e8f6f049bd906ecda0491014a6) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:02.419296 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd4452a-7824-40e2-9545-3557639af2ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:02.420751 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd4452a-7824-40e2-9545-3557639af2ae None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:02.427590 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 99/201] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:29:02 2022] GET /identity/v3/role_assignments?user.id=05d150e8f6f049bd906ecda0491014a6&scope.project.id=961f8057321f482d9a5d9202c7bc3c4c&role.id=3cda0615d87d4af0860a2a5e9b4e2c05 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:02.429399 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd4452a-7824-40e2-9545-3557639af2ae None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:02.446921 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:02.495503 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3d7934-f7ea-41bc-bfec-c013d265b0c7 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:02.538401 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:02.545034 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7bde98-f1ea-402e-9093-4f13e19c15f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:02.546466 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7bde98-f1ea-402e-9093-4f13e19c15f8 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:02.546972 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7bde98-f1ea-402e-9093-4f13e19c15f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:02.547544 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7bde98-f1ea-402e-9093-4f13e19c15f8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:02.567508 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7bde98-f1ea-402e-9093-4f13e19c15f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:02.574626 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-3a7bde98-f1ea-402e-9093-4f13e19c15f8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:29:02.574626 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:02.574626 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:02.574626 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:02.574626 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:02.574626 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:02.574626 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:02.574626 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:02.574626 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:02.574626 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:02.574626 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:02.574626 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:02.574626 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:02.574626 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:02.574626 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:02.574626 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:02.576499 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:02.576499 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:02.576499 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:02.576499 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:02.576499 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:02.576499 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:02.576499 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:02.576499 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:02.576499 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:02.576499 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:02.576499 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:02.576499 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:02.576499 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:02.576499 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:02.576499 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:02.576499 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:02.577912 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:02.577912 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:02.577912 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:02.577912 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:02.577912 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:02.577912 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:02.577912 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:02.577912 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:02.577912 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:02.577912 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:02.577912 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:29:02.577912 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:02.577912 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 103/202] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:29:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:02.582944 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7bde98-f1ea-402e-9093-4f13e19c15f8 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:02.586104 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd4452a-7824-40e2-9545-3557639af2ae None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:02.631774 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f02dba-7991-409b-b6f5-084b586aa1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:02.632745 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f02dba-7991-409b-b6f5-084b586aa1ee None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:02.633330 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f02dba-7991-409b-b6f5-084b586aa1ee None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:02.633943 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f02dba-7991-409b-b6f5-084b586aa1ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:02.634760 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f02dba-7991-409b-b6f5-084b586aa1ee None admin] RBAC: Adding query filter params (role.id=1c4886a7a7fd402d8221a0158ddcb167, scope.project.id=961f8057321f482d9a5d9202c7bc3c4c, user.id=05d150e8f6f049bd906ecda0491014a6) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:02.635141 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f02dba-7991-409b-b6f5-084b586aa1ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:02.636750 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f02dba-7991-409b-b6f5-084b586aa1ee None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:02.643789 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 100/203] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:29:02 2022] GET /identity/v3/role_assignments?user.id=05d150e8f6f049bd906ecda0491014a6&scope.project.id=961f8057321f482d9a5d9202c7bc3c4c&role.id=1c4886a7a7fd402d8221a0158ddcb167 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:02.705976 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7bde98-f1ea-402e-9093-4f13e19c15f8 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:02.751808 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea7c274-b66c-4424-ba33-47a6b16f33cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:02.752704 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7c274-b66c-4424-ba33-47a6b16f33cf None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:02.753141 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7c274-b66c-4424-ba33-47a6b16f33cf None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:02.753701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7c274-b66c-4424-ba33-47a6b16f33cf None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:02.754521 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea7c274-b66c-4424-ba33-47a6b16f33cf None admin] RBAC: Adding query filter params (name=reader) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:02.754980 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea7c274-b66c-4424-ba33-47a6b16f33cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:02.756539 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea7c274-b66c-4424-ba33-47a6b16f33cf None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:02.770343 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 104/204] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 30 14:29:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:29:02.776341 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f02dba-7991-409b-b6f5-084b586aa1ee None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:02.873599 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:02.910481 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f02dba-7991-409b-b6f5-084b586aa1ee None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:02.951410 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a2e735-e127-420a-85cf-334fcbbac662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:02.952081 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2e735-e127-420a-85cf-334fcbbac662 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:02.952365 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2e735-e127-420a-85cf-334fcbbac662 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:02.952643 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2e735-e127-420a-85cf-334fcbbac662 None admin] PATH_INFO: `/v3/users/d6285aeb36494642a235b3b6c197315a` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:02.953811 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a2e735-e127-420a-85cf-334fcbbac662 None admin] RBAC: Authorizing `identity:get_user(user_id=d6285aeb36494642a235b3b6c197315a)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:02.955294 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a2e735-e127-420a-85cf-334fcbbac662 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:02.957210 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 101/205] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:29:02 2022] GET /identity/v3/users/d6285aeb36494642a235b3b6c197315a => generated 289 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:02.962597 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea7c274-b66c-4424-ba33-47a6b16f33cf None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:03.051410 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:03.118132 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea7c274-b66c-4424-ba33-47a6b16f33cf None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:03.157977 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9853b9cc-f5f9-4f4d-8c2c-fdf76f96a744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:03.158985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9853b9cc-f5f9-4f4d-8c2c-fdf76f96a744 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:03.159506 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9853b9cc-f5f9-4f4d-8c2c-fdf76f96a744 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:03.159978 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9853b9cc-f5f9-4f4d-8c2c-fdf76f96a744 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:03.241086 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9853b9cc-f5f9-4f4d-8c2c-fdf76f96a744 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:03.242643 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9853b9cc-f5f9-4f4d-8c2c-fdf76f96a744 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:03.307629 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:03.357250 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 105/206] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:29:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:29:03.362734 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a2e735-e127-420a-85cf-334fcbbac662 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:03.492864 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a2e735-e127-420a-85cf-334fcbbac662 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:03.534724 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5982d730-74e8-4557-a3ad-d35767196ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:03.535403 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5982d730-74e8-4557-a3ad-d35767196ffc None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:03.535688 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5982d730-74e8-4557-a3ad-d35767196ffc None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:03.535969 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5982d730-74e8-4557-a3ad-d35767196ffc None admin] PATH_INFO: `/v3/projects/1c2139beef6b44b9a0574047298c1af9` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:03.537115 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5982d730-74e8-4557-a3ad-d35767196ffc None admin] RBAC: Authorizing `identity:get_project(project_id=1c2139beef6b44b9a0574047298c1af9)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:03.538571 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5982d730-74e8-4557-a3ad-d35767196ffc None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:03.540413 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 102/207] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:29:03 2022] GET /identity/v3/projects/1c2139beef6b44b9a0574047298c1af9 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:03.547193 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9853b9cc-f5f9-4f4d-8c2c-fdf76f96a744 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:03.673053 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9853b9cc-f5f9-4f4d-8c2c-fdf76f96a744 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:03.714614 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcba002e-3de9-40fa-aa2f-970688a3b6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:03.715255 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba002e-3de9-40fa-aa2f-970688a3b6ba None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:03.715603 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba002e-3de9-40fa-aa2f-970688a3b6ba None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:03.715894 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba002e-3de9-40fa-aa2f-970688a3b6ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:03.716624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcba002e-3de9-40fa-aa2f-970688a3b6ba None admin] RBAC: Adding query filter params (role.id=f51dfe78039041e3a947ea27cfe97651, scope.project.id=1c2139beef6b44b9a0574047298c1af9, user.id=d6285aeb36494642a235b3b6c197315a) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:03.716926 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcba002e-3de9-40fa-aa2f-970688a3b6ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:03.718367 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcba002e-3de9-40fa-aa2f-970688a3b6ba None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:03.727220 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 106/208] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:29:03 2022] GET /identity/v3/role_assignments?user.id=d6285aeb36494642a235b3b6c197315a&scope.project.id=1c2139beef6b44b9a0574047298c1af9&role.id=f51dfe78039041e3a947ea27cfe97651 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:04.592697 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c85be-307a-4101-9a97-412a74424e5e None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:04.593239 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c85be-307a-4101-9a97-412a74424e5e None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:04.593239 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c85be-307a-4101-9a97-412a74424e5e None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:04.594595 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 103/209] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:04.600044 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f34c46-1720-46c8-958a-7a7db1d21340 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:04.600854 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f34c46-1720-46c8-958a-7a7db1d21340 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:04.601330 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f34c46-1720-46c8-958a-7a7db1d21340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:04.620195 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-c8f34c46-1720-46c8-958a-7a7db1d21340 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:04.621716 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-c8f34c46-1720-46c8-958a-7a7db1d21340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:04.633251 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:04.691930 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 107/210] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:29:04.755330 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f7f98-085f-440f-a6f5-35a0c0c6920f None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:04.755330 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f7f98-085f-440f-a6f5-35a0c0c6920f None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:04.755330 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f7f98-085f-440f-a6f5-35a0c0c6920f None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:04.755330 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 104/211] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:04.760849 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c24b65f-3ab9-4c26-be48-48559e8b6ffd None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:04.761290 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c24b65f-3ab9-4c26-be48-48559e8b6ffd None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:04.761710 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c24b65f-3ab9-4c26-be48-48559e8b6ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:04.780928 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-4c24b65f-3ab9-4c26-be48-48559e8b6ffd None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:04.782356 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-4c24b65f-3ab9-4c26-be48-48559e8b6ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:04.853382 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 108/212] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:04.899193 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4f020-9104-42af-b103-227e066b8377 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:04.899858 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4f020-9104-42af-b103-227e066b8377 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:04.900341 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4f020-9104-42af-b103-227e066b8377 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:04.901918 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 105/213] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:04.907279 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d0860-a2ab-479d-bcc8-8669760b026a None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:04.907792 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d0860-a2ab-479d-bcc8-8669760b026a None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:04.908263 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d0860-a2ab-479d-bcc8-8669760b026a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:04.927447 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-ed1d0860-a2ab-479d-bcc8-8669760b026a None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:04.928936 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-ed1d0860-a2ab-479d-bcc8-8669760b026a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:04.954573 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc4212-a11b-4cf8-8160-9dd2c8bd57d4 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:04.955170 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc4212-a11b-4cf8-8160-9dd2c8bd57d4 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:04.955749 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc4212-a11b-4cf8-8160-9dd2c8bd57d4 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:04.958670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 106/214] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:29:04.962843 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02048e3-04b6-4744-a817-cf02e4bfb238 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:04.963202 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02048e3-04b6-4744-a817-cf02e4bfb238 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:04.963651 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02048e3-04b6-4744-a817-cf02e4bfb238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:04.982801 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-a02048e3-04b6-4744-a817-cf02e4bfb238 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:04.984066 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-a02048e3-04b6-4744-a817-cf02e4bfb238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:04.986275 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:05.002454 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 109/215] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:05.003197 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d0860-a2ab-479d-bcc8-8669760b026a None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:05.011423 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:05.074316 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 107/216] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:29:05.120198 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d0860-a2ab-479d-bcc8-8669760b026a None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:05.137301 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02048e3-04b6-4744-a817-cf02e4bfb238 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:05.157383 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538326ba-f401-49ee-8696-69c2485c79c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:05.158017 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538326ba-f401-49ee-8696-69c2485c79c6 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:05.158298 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538326ba-f401-49ee-8696-69c2485c79c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:05.158574 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538326ba-f401-49ee-8696-69c2485c79c6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:05.169686 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538326ba-f401-49ee-8696-69c2485c79c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:05.174566 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-538326ba-f401-49ee-8696-69c2485c79c6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:29:05.174566 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:05.174566 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:05.174566 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:05.174566 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:05.174566 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:05.174566 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:05.174566 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:05.174566 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:05.174566 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:05.174566 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:05.174566 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:05.174566 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:05.174566 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:05.174566 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:05.174566 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:05.175474 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:05.175474 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:05.175474 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:05.175474 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:05.175474 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:05.175474 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:05.175474 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:05.175474 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:05.175474 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:05.175474 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:05.175474 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:05.175474 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:05.175474 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:05.175474 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:05.175474 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:05.175474 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:05.176289 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:05.176289 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:05.176289 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:05.176289 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:05.176289 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:05.176289 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:05.176289 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:05.176289 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:05.176289 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:05.176289 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:05.176289 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:29:05.176289 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:05.176289 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 110/217] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:29:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:05.181519 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538326ba-f401-49ee-8696-69c2485c79c6 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:05.259624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:05.272215 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02048e3-04b6-4744-a817-cf02e4bfb238 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:05.296709 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538326ba-f401-49ee-8696-69c2485c79c6 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:05.311990 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca1137-677d-46ee-a731-55e63fb71fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:05.312622 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca1137-677d-46ee-a731-55e63fb71fbc None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:05.312907 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca1137-677d-46ee-a731-55e63fb71fbc None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:05.313188 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca1137-677d-46ee-a731-55e63fb71fbc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:05.320824 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:05.326683 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ca1137-677d-46ee-a731-55e63fb71fbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:05.331807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-14ca1137-677d-46ee-a731-55e63fb71fbc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:29:05.331807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:05.331807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:05.331807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:05.331807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:05.331807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:05.331807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:05.331807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:05.331807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:05.331807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:05.331807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:05.331807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:05.331807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:05.331807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:05.331807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:05.331807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:05.332692 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:05.332692 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:05.332692 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:05.332692 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:05.332692 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:05.332692 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:05.332692 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:05.332692 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:05.332692 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:05.332692 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:05.332692 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:05.332692 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:05.332692 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:05.332692 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:05.332692 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:05.332692 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:05.333491 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:05.333491 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:05.333491 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:05.333491 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:05.333491 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:05.333491 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:05.333491 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:05.333491 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:05.333491 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:05.333491 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:05.333491 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:29:05.333491 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:05.333491 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 108/218] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 30 14:29:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:29:05.336014 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca1137-677d-46ee-a731-55e63fb71fbc None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:05.338321 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f205e-d473-4499-bbfb-50f6c212c993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:05.338928 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f205e-d473-4499-bbfb-50f6c212c993 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:05.339638 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f205e-d473-4499-bbfb-50f6c212c993 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:05.339908 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f205e-d473-4499-bbfb-50f6c212c993 None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:05.341506 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587f205e-d473-4499-bbfb-50f6c212c993 None admin] RBAC: Adding query filter params (name=member) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:05.341812 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587f205e-d473-4499-bbfb-50f6c212c993 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:05.343177 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587f205e-d473-4499-bbfb-50f6c212c993 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:05.352490 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 111/219] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 30 14:29:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:29:05.353202 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f205e-d473-4499-bbfb-50f6c212c993 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:05.443115 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca1137-677d-46ee-a731-55e63fb71fbc None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:05.455992 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f205e-d473-4499-bbfb-50f6c212c993 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:05.476015 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd53988-5f4e-4367-ac09-998451427357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:05.476640 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd53988-5f4e-4367-ac09-998451427357 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:05.476917 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd53988-5f4e-4367-ac09-998451427357 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:05.477186 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd53988-5f4e-4367-ac09-998451427357 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:05.478280 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd53988-5f4e-4367-ac09-998451427357 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:05.479787 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd53988-5f4e-4367-ac09-998451427357 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:05.481377 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 109/220] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:29:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:05.483378 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd53988-5f4e-4367-ac09-998451427357 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:05.483662 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:05.492573 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673d56f7-8815-41d3-872a-c8c3a021de50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:05.493211 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d56f7-8815-41d3-872a-c8c3a021de50 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:05.493495 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d56f7-8815-41d3-872a-c8c3a021de50 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:05.493771 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d56f7-8815-41d3-872a-c8c3a021de50 None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:05.494453 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673d56f7-8815-41d3-872a-c8c3a021de50 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:05.494855 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673d56f7-8815-41d3-872a-c8c3a021de50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:05.496284 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673d56f7-8815-41d3-872a-c8c3a021de50 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:05.505066 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 112/221] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:29:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:05.505708 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673d56f7-8815-41d3-872a-c8c3a021de50 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:05.551080 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:05.586826 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd53988-5f4e-4367-ac09-998451427357 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:05.606807 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673d56f7-8815-41d3-872a-c8c3a021de50 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:05.620486 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138a9180-4c71-4603-8d8d-6c32281e0529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:05.621109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a9180-4c71-4603-8d8d-6c32281e0529 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:05.621389 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a9180-4c71-4603-8d8d-6c32281e0529 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:05.622191 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a9180-4c71-4603-8d8d-6c32281e0529 None admin] PATH_INFO: `/v3/users/10ea7abdff734626b2ffe657de881f08` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:05.635145 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138a9180-4c71-4603-8d8d-6c32281e0529 None admin] RBAC: Authorizing `identity:get_user(user_id=10ea7abdff734626b2ffe657de881f08)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:05.636625 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138a9180-4c71-4603-8d8d-6c32281e0529 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:05.638523 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 110/222] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:29:05 2022] GET /identity/v3/users/10ea7abdff734626b2ffe657de881f08 => generated 297 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:05.639112 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138a9180-4c71-4603-8d8d-6c32281e0529 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:05.644464 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a96b8d-ee39-4c46-80b9-e26800dbc411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:05.645108 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a96b8d-ee39-4c46-80b9-e26800dbc411 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:05.645390 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a96b8d-ee39-4c46-80b9-e26800dbc411 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:05.645667 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a96b8d-ee39-4c46-80b9-e26800dbc411 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:05.656553 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a96b8d-ee39-4c46-80b9-e26800dbc411 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:05.663460 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-e3a96b8d-ee39-4c46-80b9-e26800dbc411 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:29:05.663460 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:05.663460 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:05.663460 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:05.663460 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:05.663460 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:05.663460 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:05.663460 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:05.663460 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:05.663460 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:05.663460 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:05.663460 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:05.663460 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:05.663460 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:05.663460 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:05.663460 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:05.664354 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:05.664354 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:05.664354 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:05.664354 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:05.664354 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:05.664354 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:05.664354 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:05.664354 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:05.664354 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:05.664354 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:05.664354 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:05.664354 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:05.664354 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:05.664354 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:05.664354 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:05.664354 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:05.665195 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:05.665195 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:05.665195 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:05.665195 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:05.665195 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:05.665195 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:05.665195 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:05.665195 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:05.665195 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:05.665195 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:05.665195 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:29:05.665195 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:05.665804 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 113/223] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 30 14:29:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:05.666913 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a96b8d-ee39-4c46-80b9-e26800dbc411 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:05.720926 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:05.743798 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138a9180-4c71-4603-8d8d-6c32281e0529 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:05.771860 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a96b8d-ee39-4c46-80b9-e26800dbc411 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:05.773885 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:05.786030 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57acf83f-a210-49d1-ade6-4e1279702007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:05.786636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57acf83f-a210-49d1-ade6-4e1279702007 None admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:05.786916 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57acf83f-a210-49d1-ade6-4e1279702007 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:05.787187 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57acf83f-a210-49d1-ade6-4e1279702007 None admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:05.787869 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57acf83f-a210-49d1-ade6-4e1279702007 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:05.789340 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57acf83f-a210-49d1-ade6-4e1279702007 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:05.805816 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b7efcf-ec47-40d3-bc3a-2d6c4286fd00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:05.806450 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b7efcf-ec47-40d3-bc3a-2d6c4286fd00 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:05.806818 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b7efcf-ec47-40d3-bc3a-2d6c4286fd00 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:05.807156 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b7efcf-ec47-40d3-bc3a-2d6c4286fd00 None admin] PATH_INFO: `/v3/users/05d150e8f6f049bd906ecda0491014a6` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:05.808334 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b7efcf-ec47-40d3-bc3a-2d6c4286fd00 None admin] RBAC: Authorizing `identity:get_user(user_id=05d150e8f6f049bd906ecda0491014a6)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:05.810323 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b7efcf-ec47-40d3-bc3a-2d6c4286fd00 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:05.810806 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 111/224] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 30 14:29:05 2022] POST /identity/v3/users => generated 297 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:29:05.811667 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57acf83f-a210-49d1-ade6-4e1279702007 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:05.826263 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 114/225] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:29:05 2022] GET /identity/v3/users/05d150e8f6f049bd906ecda0491014a6 => generated 283 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:05.826263 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b7efcf-ec47-40d3-bc3a-2d6c4286fd00 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:05.927281 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57acf83f-a210-49d1-ade6-4e1279702007 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:05.952749 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b7efcf-ec47-40d3-bc3a-2d6c4286fd00 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:05.968451 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d62b0-3617-472b-a7ff-2c613f73b8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:05.969073 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d62b0-3617-472b-a7ff-2c613f73b8ea None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:05.969355 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d62b0-3617-472b-a7ff-2c613f73b8ea None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:05.970386 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d62b0-3617-472b-a7ff-2c613f73b8ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:05.975812 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:06.002088 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bb220d-723d-42eb-9aa4-13ddc7a95a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:06.002702 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb220d-723d-42eb-9aa4-13ddc7a95a16 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:06.002982 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb220d-723d-42eb-9aa4-13ddc7a95a16 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:06.003287 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb220d-723d-42eb-9aa4-13ddc7a95a16 None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:06.003955 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bb220d-723d-42eb-9aa4-13ddc7a95a16 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:06.004256 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bb220d-723d-42eb-9aa4-13ddc7a95a16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:06.005642 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bb220d-723d-42eb-9aa4-13ddc7a95a16 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:06.016488 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 115/226] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:29:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:06.017465 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bb220d-723d-42eb-9aa4-13ddc7a95a16 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:06.025307 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:06.055281 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d62b0-3617-472b-a7ff-2c613f73b8ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:06.057071 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d62b0-3617-472b-a7ff-2c613f73b8ea None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:06.147153 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bb220d-723d-42eb-9aa4-13ddc7a95a16 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:06.185008 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb4b020-b197-42c3-9ef0-e5bae9ff1448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:06.185740 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb4b020-b197-42c3-9ef0-e5bae9ff1448 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:06.186015 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb4b020-b197-42c3-9ef0-e5bae9ff1448 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:06.186287 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb4b020-b197-42c3-9ef0-e5bae9ff1448 None admin] PATH_INFO: `/v3/projects/961f8057321f482d9a5d9202c7bc3c4c` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:06.187122 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 112/227] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:29:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:29:06.187783 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb4b020-b197-42c3-9ef0-e5bae9ff1448 None admin] RBAC: Authorizing `identity:get_project(project_id=961f8057321f482d9a5d9202c7bc3c4c)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:06.187974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d62b0-3617-472b-a7ff-2c613f73b8ea None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:06.189516 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb4b020-b197-42c3-9ef0-e5bae9ff1448 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:06.191465 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 116/228] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:29:06 2022] GET /identity/v3/projects/961f8057321f482d9a5d9202c7bc3c4c => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:06.201087 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb4b020-b197-42c3-9ef0-e5bae9ff1448 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:06.270499 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:06.289996 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:06.320255 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d62b0-3617-472b-a7ff-2c613f73b8ea None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:06.335836 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb4b020-b197-42c3-9ef0-e5bae9ff1448 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:06.366804 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ccbf6a-b9d2-4821-bac4-ab8586eae2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:06.367936 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ccbf6a-b9d2-4821-bac4-ab8586eae2dc None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:06.368401 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ccbf6a-b9d2-4821-bac4-ab8586eae2dc None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:06.368811 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ccbf6a-b9d2-4821-bac4-ab8586eae2dc None admin] PATH_INFO: `/v3/users/05d150e8f6f049bd906ecda0491014a6` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:06.370092 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ccbf6a-b9d2-4821-bac4-ab8586eae2dc None admin] RBAC: Authorizing `identity:get_user(user_id=05d150e8f6f049bd906ecda0491014a6)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:06.371182 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0105ebb0-eb0f-44e0-ad1e-600f05c56682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:06.372463 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ccbf6a-b9d2-4821-bac4-ab8586eae2dc None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:06.372645 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0105ebb0-eb0f-44e0-ad1e-600f05c56682 None admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:06.372795 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0105ebb0-eb0f-44e0-ad1e-600f05c56682 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:06.373058 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0105ebb0-eb0f-44e0-ad1e-600f05c56682 None admin] PATH_INFO: `/v3/projects/961f8057321f482d9a5d9202c7bc3c4c/users/05d150e8f6f049bd906ecda0491014a6/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:06.374805 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 113/229] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:29:06 2022] GET /identity/v3/users/05d150e8f6f049bd906ecda0491014a6 => generated 283 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:06.374973 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0105ebb0-eb0f-44e0-ad1e-600f05c56682 None admin] RBAC: Authorizing `identity:create_grant(project_id=961f8057321f482d9a5d9202c7bc3c4c, user_id=05d150e8f6f049bd906ecda0491014a6, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:06.375865 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ccbf6a-b9d2-4821-bac4-ab8586eae2dc None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:06.376523 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0105ebb0-eb0f-44e0-ad1e-600f05c56682 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:06.393474 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 117/230] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 30 14:29:06 2022] PUT /identity/v3/projects/961f8057321f482d9a5d9202c7bc3c4c/users/05d150e8f6f049bd906ecda0491014a6/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:29:06.400419 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0105ebb0-eb0f-44e0-ad1e-600f05c56682 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:06.546034 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ccbf6a-b9d2-4821-bac4-ab8586eae2dc None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:06.587944 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0105ebb0-eb0f-44e0-ad1e-600f05c56682 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:06.599092 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:06.600060 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a2194-4bbc-4ebd-99bf-26bb1f00ad85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:06.601422 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a2194-4bbc-4ebd-99bf-26bb1f00ad85 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:06.601703 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a2194-4bbc-4ebd-99bf-26bb1f00ad85 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:06.602540 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a2194-4bbc-4ebd-99bf-26bb1f00ad85 None admin] PATH_INFO: `/v3/projects/961f8057321f482d9a5d9202c7bc3c4c` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:06.604972 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980a2194-4bbc-4ebd-99bf-26bb1f00ad85 None admin] RBAC: Authorizing `identity:get_project(project_id=961f8057321f482d9a5d9202c7bc3c4c)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:06.608871 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980a2194-4bbc-4ebd-99bf-26bb1f00ad85 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:06.611129 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 114/231] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:29:06 2022] GET /identity/v3/projects/961f8057321f482d9a5d9202c7bc3c4c => generated 302 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:06.617221 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a2194-4bbc-4ebd-99bf-26bb1f00ad85 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:06.639134 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a20c9-94f5-465c-b77a-5a44de4b817e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:06.639700 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:06.640031 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a20c9-94f5-465c-b77a-5a44de4b817e None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:06.640126 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a20c9-94f5-465c-b77a-5a44de4b817e None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:06.640685 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a20c9-94f5-465c-b77a-5a44de4b817e None admin] PATH_INFO: `/v3/projects/961f8057321f482d9a5d9202c7bc3c4c` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:06.641538 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3a20c9-94f5-465c-b77a-5a44de4b817e None admin] RBAC: Authorizing `identity:get_project(project_id=961f8057321f482d9a5d9202c7bc3c4c)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:06.643371 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3a20c9-94f5-465c-b77a-5a44de4b817e None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:06.645216 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 118/232] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:29:06 2022] GET /identity/v3/projects/961f8057321f482d9a5d9202c7bc3c4c => generated 302 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:06.652078 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a20c9-94f5-465c-b77a-5a44de4b817e None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:06.785122 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a2194-4bbc-4ebd-99bf-26bb1f00ad85 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:06.820726 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a20c9-94f5-465c-b77a-5a44de4b817e None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:06.840044 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11156d91-9cb5-4b69-80d0-f2ab34835ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:06.842201 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11156d91-9cb5-4b69-80d0-f2ab34835ca7 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:06.842487 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11156d91-9cb5-4b69-80d0-f2ab34835ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:06.842761 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11156d91-9cb5-4b69-80d0-f2ab34835ca7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:06.843529 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11156d91-9cb5-4b69-80d0-f2ab34835ca7 None admin] RBAC: Adding query filter params (role.id=6a985d5659a3416cba33ca9d30b39f85, scope.project.id=961f8057321f482d9a5d9202c7bc3c4c, user.id=10ea7abdff734626b2ffe657de881f08) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:06.843837 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11156d91-9cb5-4b69-80d0-f2ab34835ca7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:06.845506 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11156d91-9cb5-4b69-80d0-f2ab34835ca7 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:06.858926 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 115/233] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:29:06 2022] GET /identity/v3/role_assignments?user.id=10ea7abdff734626b2ffe657de881f08&scope.project.id=961f8057321f482d9a5d9202c7bc3c4c&role.id=6a985d5659a3416cba33ca9d30b39f85 => generated 259 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:06.874643 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee087b-2d1d-49ef-9d63-ef9fda45adad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:06.875369 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee087b-2d1d-49ef-9d63-ef9fda45adad None admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:06.875649 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee087b-2d1d-49ef-9d63-ef9fda45adad None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:06.875920 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee087b-2d1d-49ef-9d63-ef9fda45adad None admin] PATH_INFO: `/v3/projects/961f8057321f482d9a5d9202c7bc3c4c/users/05d150e8f6f049bd906ecda0491014a6/roles/1c4886a7a7fd402d8221a0158ddcb167` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:06.878180 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaee087b-2d1d-49ef-9d63-ef9fda45adad None admin] RBAC: Authorizing `identity:create_grant(project_id=961f8057321f482d9a5d9202c7bc3c4c, user_id=05d150e8f6f049bd906ecda0491014a6, role_id=1c4886a7a7fd402d8221a0158ddcb167)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:06.879824 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaee087b-2d1d-49ef-9d63-ef9fda45adad None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:06.907330 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 119/234] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 30 14:29:06 2022] PUT /identity/v3/projects/961f8057321f482d9a5d9202c7bc3c4c/users/05d150e8f6f049bd906ecda0491014a6/roles/1c4886a7a7fd402d8221a0158ddcb167 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:29:08.133392 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c153a59-c8ba-4e16-8b0b-f6ee275fc334 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:08.134085 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c153a59-c8ba-4e16-8b0b-f6ee275fc334 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:08.134085 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c153a59-c8ba-4e16-8b0b-f6ee275fc334 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:08.135318 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 116/235] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:08.141642 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27218a76-2fad-4f22-b2c5-19bcd937a2f6 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:08.142140 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27218a76-2fad-4f22-b2c5-19bcd937a2f6 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:08.142287 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27218a76-2fad-4f22-b2c5-19bcd937a2f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:08.168218 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-27218a76-2fad-4f22-b2c5-19bcd937a2f6 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:08.169466 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-27218a76-2fad-4f22-b2c5-19bcd937a2f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:08.206837 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb96fe-1e14-4cde-8dac-212fa346fe28 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:08.207160 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb96fe-1e14-4cde-8dac-212fa346fe28 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:08.207413 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb96fe-1e14-4cde-8dac-212fa346fe28 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:08.208763 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 117/236] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:08.212770 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:08.222597 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6f728-c478-4299-b952-b1dd3b431819 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:08.222873 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6f728-c478-4299-b952-b1dd3b431819 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:08.223148 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6f728-c478-4299-b952-b1dd3b431819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:08.249287 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-7ec6f728-c478-4299-b952-b1dd3b431819 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:08.250489 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-7ec6f728-c478-4299-b952-b1dd3b431819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:08.294639 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 120/237] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:08.360774 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 118/238] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:29:08.637954 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27218a76-2fad-4f22-b2c5-19bcd937a2f6 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:08.741676 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec6f728-c478-4299-b952-b1dd3b431819 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:08.758761 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27218a76-2fad-4f22-b2c5-19bcd937a2f6 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:08.764733 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:08.833047 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdf0f1f-68e0-43e3-8ea8-3d5d340503ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:08.833692 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf0f1f-68e0-43e3-8ea8-3d5d340503ad None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:08.833968 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf0f1f-68e0-43e3-8ea8-3d5d340503ad None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:08.834237 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf0f1f-68e0-43e3-8ea8-3d5d340503ad None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:08.853945 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdf0f1f-68e0-43e3-8ea8-3d5d340503ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:08.860773 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-7cdf0f1f-68e0-43e3-8ea8-3d5d340503ad None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:29:08.860773 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:08.860773 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:08.860773 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:08.860773 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:08.860773 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:08.860773 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:08.860773 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:08.860773 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:08.860773 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:08.860773 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:08.860773 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:08.860773 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:08.860773 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:08.860773 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:08.860773 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:08.861672 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:08.861672 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:08.861672 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:08.861672 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:08.861672 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:08.861672 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:08.861672 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:08.861672 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:08.861672 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:08.861672 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:08.861672 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:08.861672 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:08.861672 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:08.861672 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:08.861672 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:08.861672 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:08.862469 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:08.862469 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:08.862469 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:08.862469 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:08.862469 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:08.862469 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:08.862469 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:08.862469 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:08.862469 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:08.862469 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:08.862469 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:29:08.862469 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:08.863603 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 121/239] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:29:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:08.871014 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdf0f1f-68e0-43e3-8ea8-3d5d340503ad None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:08.928894 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:08.945183 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec6f728-c478-4299-b952-b1dd3b431819 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:08.983262 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d86c0d-2e19-4f5a-a610-709fddd21a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:08.983902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d86c0d-2e19-4f5a-a610-709fddd21a0d None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:08.984216 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d86c0d-2e19-4f5a-a610-709fddd21a0d None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:08.984497 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d86c0d-2e19-4f5a-a610-709fddd21a0d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:08.985604 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d86c0d-2e19-4f5a-a610-709fddd21a0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:08.987098 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d86c0d-2e19-4f5a-a610-709fddd21a0d None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:08.988829 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 119/240] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:29:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:08.989949 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9d811-1d32-40b1-920b-981d722638cd None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:08.990226 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9d811-1d32-40b1-920b-981d722638cd None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:08.990498 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9d811-1d32-40b1-920b-981d722638cd None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:08.992254 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 120/241] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:08.999926 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f9d811-1d32-40b1-920b-981d722638cd None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:09.023941 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdf0f1f-68e0-43e3-8ea8-3d5d340503ad None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:09.067735 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f28500-e5eb-4889-a0c0-843db950ed3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:09.068373 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f28500-e5eb-4889-a0c0-843db950ed3f None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:09.068752 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f28500-e5eb-4889-a0c0-843db950ed3f None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:09.069025 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f28500-e5eb-4889-a0c0-843db950ed3f None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:09.069789 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f28500-e5eb-4889-a0c0-843db950ed3f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:09.070149 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f28500-e5eb-4889-a0c0-843db950ed3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:09.071622 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f28500-e5eb-4889-a0c0-843db950ed3f None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:09.088597 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 122/242] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 30 14:29:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:29:09.090016 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5eada4-4aa0-483b-8293-49a1d2aba7c7 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:09.090357 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5eada4-4aa0-483b-8293-49a1d2aba7c7 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:09.091130 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5eada4-4aa0-483b-8293-49a1d2aba7c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:09.105882 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:09.113302 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-dc5eada4-4aa0-483b-8293-49a1d2aba7c7 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:09.114823 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-dc5eada4-4aa0-483b-8293-49a1d2aba7c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:09.135456 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f9d811-1d32-40b1-920b-981d722638cd None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:09.184824 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455784a4-fd57-4411-979b-9f9c087ade76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:09.185773 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455784a4-fd57-4411-979b-9f9c087ade76 None admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:09.186176 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455784a4-fd57-4411-979b-9f9c087ade76 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:09.186453 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455784a4-fd57-4411-979b-9f9c087ade76 None admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:09.187458 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455784a4-fd57-4411-979b-9f9c087ade76 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:09.189168 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455784a4-fd57-4411-979b-9f9c087ade76 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:09.192149 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 123/243] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:09.193501 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5eada4-4aa0-483b-8293-49a1d2aba7c7 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:09.232297 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:09.240165 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 121/244] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 30 14:29:08 2022] POST /identity/v3/users => generated 293 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:29:09.245140 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd93b3b-0e3a-4dc2-84d0-782794ce1aaf None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:09.245140 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd93b3b-0e3a-4dc2-84d0-782794ce1aaf None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:09.245140 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd93b3b-0e3a-4dc2-84d0-782794ce1aaf None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:09.245140 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 122/245] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:09.251767 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b87a759-cad0-4fd1-bb3c-6b3e4a1a7a9c None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:09.252064 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b87a759-cad0-4fd1-bb3c-6b3e4a1a7a9c None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:09.252324 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b87a759-cad0-4fd1-bb3c-6b3e4a1a7a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:09.276116 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-3b87a759-cad0-4fd1-bb3c-6b3e4a1a7a9c None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:09.277681 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-3b87a759-cad0-4fd1-bb3c-6b3e4a1a7a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:09.366486 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 123/246] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:09.372916 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd47aab-1860-4d83-9d19-bc499c47f1e0 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:09.373611 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd47aab-1860-4d83-9d19-bc499c47f1e0 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:09.374097 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd47aab-1860-4d83-9d19-bc499c47f1e0 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:09.376887 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 124/247] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:09 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:09.384819 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c02eaf-2e8b-48c5-a2a3-e8280e513321 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:09.385441 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c02eaf-2e8b-48c5-a2a3-e8280e513321 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:09.385990 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c02eaf-2e8b-48c5-a2a3-e8280e513321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:09.392982 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5eada4-4aa0-483b-8293-49a1d2aba7c7 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:09.416872 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-04c02eaf-2e8b-48c5-a2a3-e8280e513321 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:09.420217 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-04c02eaf-2e8b-48c5-a2a3-e8280e513321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:09.462646 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcc52d8-eb68-4d79-9358-791e752f4310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:09.463700 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcc52d8-eb68-4d79-9358-791e752f4310 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:09.464092 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcc52d8-eb68-4d79-9358-791e752f4310 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:09.464461 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcc52d8-eb68-4d79-9358-791e752f4310 None admin] PATH_INFO: `/v3/users/aa171c8644ee4d23babe25ebdaa1cf70` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:09.482897 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcc52d8-eb68-4d79-9358-791e752f4310 None admin] RBAC: Authorizing `identity:get_user(user_id=aa171c8644ee4d23babe25ebdaa1cf70)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:09.484129 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcc52d8-eb68-4d79-9358-791e752f4310 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:09.485818 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 124/248] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:29:09 2022] GET /identity/v3/users/aa171c8644ee4d23babe25ebdaa1cf70 => generated 297 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:09.488112 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcc52d8-eb68-4d79-9358-791e752f4310 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:09.508812 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:09.519543 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 125/249] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:09.520673 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c02eaf-2e8b-48c5-a2a3-e8280e513321 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:09.629535 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcc52d8-eb68-4d79-9358-791e752f4310 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:09.635095 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c02eaf-2e8b-48c5-a2a3-e8280e513321 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:09.639581 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:09.673947 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df9351-fd4e-48d1-b453-d6885273fc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:09.675162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df9351-fd4e-48d1-b453-d6885273fc49 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:09.675542 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df9351-fd4e-48d1-b453-d6885273fc49 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:09.677235 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df9351-fd4e-48d1-b453-d6885273fc49 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:09.689844 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273f809b-cda2-4a7c-9862-71ee5251fe6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:09.690681 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f809b-cda2-4a7c-9862-71ee5251fe6b None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:09.691032 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7df9351-fd4e-48d1-b453-d6885273fc49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:09.693464 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f809b-cda2-4a7c-9862-71ee5251fe6b None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:09.693892 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f809b-cda2-4a7c-9862-71ee5251fe6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:09.696452 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-b7df9351-fd4e-48d1-b453-d6885273fc49 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:29:09.696452 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:09.696452 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:09.696452 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:09.696452 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:09.696452 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:09.696452 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:09.696452 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:09.696452 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:09.696452 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:09.696452 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:09.696452 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:09.696452 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:09.696452 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:09.696452 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:09.696452 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:09.697332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:09.697332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:09.697332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:09.697332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:09.697332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:09.697332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:09.697332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:09.697332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:09.697332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:09.697332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:09.697332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:09.697332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:09.697332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:09.697332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:09.697332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:09.697332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:09.698118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:09.698118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:09.698118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:09.698118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:09.698118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:09.698118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:09.698118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:09.698118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:09.698118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:09.698118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:09.698118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:29:09.698118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:09.698118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 125/250] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 30 14:29:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:09.701536 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df9351-fd4e-48d1-b453-d6885273fc49 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:09.781437 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:09.784337 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273f809b-cda2-4a7c-9862-71ee5251fe6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:09.786189 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273f809b-cda2-4a7c-9862-71ee5251fe6b None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:09.843306 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df9351-fd4e-48d1-b453-d6885273fc49 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:09.889005 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d650e645-43eb-4b26-b4f4-48c85f5c6702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:09.889967 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d650e645-43eb-4b26-b4f4-48c85f5c6702 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:09.890247 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d650e645-43eb-4b26-b4f4-48c85f5c6702 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:09.890519 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d650e645-43eb-4b26-b4f4-48c85f5c6702 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:09.904901 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d650e645-43eb-4b26-b4f4-48c85f5c6702 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:09.905746 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 126/251] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:29:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:29:09.906421 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273f809b-cda2-4a7c-9862-71ee5251fe6b None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:09.912398 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-d650e645-43eb-4b26-b4f4-48c85f5c6702 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:29:09.912398 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:09.912398 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:09.912398 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:09.912398 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:09.912398 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:09.912398 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:09.912398 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:09.912398 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:09.912398 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:09.912398 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:09.912398 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:09.912398 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:09.912398 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:09.912398 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:09.912398 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:09.913290 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:09.913290 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:09.913290 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:09.913290 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:09.913290 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:09.913290 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:09.913290 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:09.913290 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:09.913290 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:09.913290 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:09.913290 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:09.913290 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:09.913290 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:09.913290 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:09.913290 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:09.913290 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:09.914093 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:09.914093 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:09.914093 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:09.914093 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:09.914093 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:09.914093 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:09.914093 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:09.914093 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:09.914093 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:09.914093 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:09.914093 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:29:09.914093 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:09.914093 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 126/252] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:29:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:09.914739 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d650e645-43eb-4b26-b4f4-48c85f5c6702 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:09.915685 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:10.019439 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:10.027376 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273f809b-cda2-4a7c-9862-71ee5251fe6b None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:10.049213 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d650e645-43eb-4b26-b4f4-48c85f5c6702 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:10.067358 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ca8b0c-a7f3-4fe7-a130-953336c4531b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:10.068111 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca8b0c-a7f3-4fe7-a130-953336c4531b None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:10.068488 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca8b0c-a7f3-4fe7-a130-953336c4531b None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:10.068912 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca8b0c-a7f3-4fe7-a130-953336c4531b None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:10.069678 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ca8b0c-a7f3-4fe7-a130-953336c4531b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:10.070093 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ca8b0c-a7f3-4fe7-a130-953336c4531b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:10.071588 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ca8b0c-a7f3-4fe7-a130-953336c4531b None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:10.085519 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 127/253] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:29:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:10.085519 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ca8b0c-a7f3-4fe7-a130-953336c4531b None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:10.101836 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0da81b-4c6f-48ed-9c57-ef490931c7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:10.103154 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0da81b-4c6f-48ed-9c57-ef490931c7a4 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:10.103615 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0da81b-4c6f-48ed-9c57-ef490931c7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:10.104299 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0da81b-4c6f-48ed-9c57-ef490931c7a4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:10.118021 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0da81b-4c6f-48ed-9c57-ef490931c7a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:10.123595 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-3b0da81b-4c6f-48ed-9c57-ef490931c7a4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:29:10.123595 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:10.123595 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:10.123595 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:10.123595 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:10.123595 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:10.123595 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:10.123595 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:10.123595 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:10.123595 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:10.123595 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:10.123595 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:10.123595 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:10.123595 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:10.123595 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:10.123595 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:10.124697 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:10.124697 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:10.124697 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:10.124697 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:10.124697 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:10.124697 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:10.124697 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:10.124697 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:10.124697 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:10.124697 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:10.124697 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:10.124697 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:10.124697 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:10.124697 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:10.124697 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:10.124697 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:10.125578 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:10.125578 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:10.125578 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:10.125578 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:10.125578 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:10.125578 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:10.125578 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:10.125578 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:10.125578 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:10.125578 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:10.125578 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:29:10.125578 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:10.125578 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 127/254] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 30 14:29:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:10.125578 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0da81b-4c6f-48ed-9c57-ef490931c7a4 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:10.188326 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:10.221167 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ca8b0c-a7f3-4fe7-a130-953336c4531b None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:10.242268 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0da81b-4c6f-48ed-9c57-ef490931c7a4 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:10.264384 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2682af-1d59-4a38-921f-3e459744b649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:10.265540 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2682af-1d59-4a38-921f-3e459744b649 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:10.265965 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2682af-1d59-4a38-921f-3e459744b649 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:10.266377 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2682af-1d59-4a38-921f-3e459744b649 None admin] PATH_INFO: `/v3/projects/961f8057321f482d9a5d9202c7bc3c4c` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:10.268101 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2682af-1d59-4a38-921f-3e459744b649 None admin] RBAC: Authorizing `identity:get_project(project_id=961f8057321f482d9a5d9202c7bc3c4c)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:10.270669 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2682af-1d59-4a38-921f-3e459744b649 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:10.277911 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 128/255] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:29:10 2022] GET /identity/v3/projects/961f8057321f482d9a5d9202c7bc3c4c => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:10.277911 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2682af-1d59-4a38-921f-3e459744b649 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:10.285950 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d1ffdd-30a0-48de-9a52-5d2e0bc22237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:10.286580 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1ffdd-30a0-48de-9a52-5d2e0bc22237 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:10.287034 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1ffdd-30a0-48de-9a52-5d2e0bc22237 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:10.287343 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1ffdd-30a0-48de-9a52-5d2e0bc22237 None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:10.288024 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d1ffdd-30a0-48de-9a52-5d2e0bc22237 None admin] RBAC: Adding query filter params (name=member) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:10.288323 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d1ffdd-30a0-48de-9a52-5d2e0bc22237 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:10.289825 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d1ffdd-30a0-48de-9a52-5d2e0bc22237 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:10.299963 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 128/256] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 30 14:29:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:10.300820 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d1ffdd-30a0-48de-9a52-5d2e0bc22237 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:10.313791 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:10.410234 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2682af-1d59-4a38-921f-3e459744b649 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:10.412871 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d1ffdd-30a0-48de-9a52-5d2e0bc22237 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:10.431033 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:10.451542 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c56f43-87a0-4580-8f6e-ae9958b8627c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:10.452612 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c56f43-87a0-4580-8f6e-ae9958b8627c None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:10.453066 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c56f43-87a0-4580-8f6e-ae9958b8627c None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:10.453529 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c56f43-87a0-4580-8f6e-ae9958b8627c None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:10.453850 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b76b1a-d1f7-44dd-9d6e-99c11302149d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:10.454554 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b76b1a-d1f7-44dd-9d6e-99c11302149d None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:10.454757 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c56f43-87a0-4580-8f6e-ae9958b8627c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:10.454905 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b76b1a-d1f7-44dd-9d6e-99c11302149d None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:10.455328 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b76b1a-d1f7-44dd-9d6e-99c11302149d None admin] PATH_INFO: `/v3/users/05d150e8f6f049bd906ecda0491014a6` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:10.455652 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c56f43-87a0-4580-8f6e-ae9958b8627c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:10.456471 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b76b1a-d1f7-44dd-9d6e-99c11302149d None admin] RBAC: Authorizing `identity:get_user(user_id=05d150e8f6f049bd906ecda0491014a6)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:10.457555 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c56f43-87a0-4580-8f6e-ae9958b8627c None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:10.457910 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b76b1a-d1f7-44dd-9d6e-99c11302149d None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:10.459707 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 129/257] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:29:10 2022] GET /identity/v3/users/05d150e8f6f049bd906ecda0491014a6 => generated 283 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:10.460909 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b76b1a-d1f7-44dd-9d6e-99c11302149d None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:10.467962 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 129/258] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:29:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:10.469109 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c56f43-87a0-4580-8f6e-ae9958b8627c None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:10.575010 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:10.585328 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b76b1a-d1f7-44dd-9d6e-99c11302149d None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:10.585640 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c56f43-87a0-4580-8f6e-ae9958b8627c None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:10.620681 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871ce402-ada6-43bc-b7ac-f7a8cf371679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:10.621302 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ce402-ada6-43bc-b7ac-f7a8cf371679 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:10.622125 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ce402-ada6-43bc-b7ac-f7a8cf371679 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:10.622403 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ce402-ada6-43bc-b7ac-f7a8cf371679 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:10.623197 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871ce402-ada6-43bc-b7ac-f7a8cf371679 None admin] RBAC: Adding query filter params (role.id=f51dfe78039041e3a947ea27cfe97651, scope.project.id=961f8057321f482d9a5d9202c7bc3c4c, user.id=aa171c8644ee4d23babe25ebdaa1cf70) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:10.623553 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871ce402-ada6-43bc-b7ac-f7a8cf371679 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:10.625207 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871ce402-ada6-43bc-b7ac-f7a8cf371679 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:10.625769 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1006d640-8b66-4ae6-a30c-daf62dc2b5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:10.626610 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1006d640-8b66-4ae6-a30c-daf62dc2b5dd None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:10.627133 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1006d640-8b66-4ae6-a30c-daf62dc2b5dd None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:10.627729 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1006d640-8b66-4ae6-a30c-daf62dc2b5dd None admin] PATH_INFO: `/v3/users/10ea7abdff734626b2ffe657de881f08` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:10.629105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1006d640-8b66-4ae6-a30c-daf62dc2b5dd None admin] RBAC: Authorizing `identity:get_user(user_id=10ea7abdff734626b2ffe657de881f08)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:10.630898 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 130/259] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:29:10 2022] GET /identity/v3/role_assignments?user.id=aa171c8644ee4d23babe25ebdaa1cf70&scope.project.id=961f8057321f482d9a5d9202c7bc3c4c&role.id=f51dfe78039041e3a947ea27cfe97651 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:10.631301 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1006d640-8b66-4ae6-a30c-daf62dc2b5dd None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:10.633195 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 130/260] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:29:10 2022] GET /identity/v3/users/10ea7abdff734626b2ffe657de881f08 => generated 297 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:10.634369 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1006d640-8b66-4ae6-a30c-daf62dc2b5dd None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:10.635621 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871ce402-ada6-43bc-b7ac-f7a8cf371679 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:10.737127 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:10.841474 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1006d640-8b66-4ae6-a30c-daf62dc2b5dd None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:10.862501 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871ce402-ada6-43bc-b7ac-f7a8cf371679 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:10.912254 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aedcaf-07c0-441f-8065-5543fc278ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:10.913554 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aedcaf-07c0-441f-8065-5543fc278ab5 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:10.913839 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aedcaf-07c0-441f-8065-5543fc278ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:10.914115 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aedcaf-07c0-441f-8065-5543fc278ab5 None admin] PATH_INFO: `/v3/users/05d150e8f6f049bd906ecda0491014a6` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:10.915417 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aedcaf-07c0-441f-8065-5543fc278ab5 None admin] RBAC: Authorizing `identity:get_user(user_id=05d150e8f6f049bd906ecda0491014a6)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:10.916883 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aedcaf-07c0-441f-8065-5543fc278ab5 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:10.919363 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 131/261] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:29:10 2022] GET /identity/v3/users/05d150e8f6f049bd906ecda0491014a6 => generated 283 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:10.919856 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aedcaf-07c0-441f-8065-5543fc278ab5 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:10.928709 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5119581c-d050-4b3f-a2b2-97f62643e4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:10.929330 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5119581c-d050-4b3f-a2b2-97f62643e4bf None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:10.929615 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5119581c-d050-4b3f-a2b2-97f62643e4bf None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:10.929896 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5119581c-d050-4b3f-a2b2-97f62643e4bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:10.979849 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:11.028821 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5119581c-d050-4b3f-a2b2-97f62643e4bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:11.030449 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5119581c-d050-4b3f-a2b2-97f62643e4bf None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:11.129425 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aedcaf-07c0-441f-8065-5543fc278ab5 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:11.150532 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 131/262] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:29:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:29:11.151188 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5119581c-d050-4b3f-a2b2-97f62643e4bf None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:11.171943 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:11.203965 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf2e772-5667-474b-8a5a-8044c8617524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:11.204616 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf2e772-5667-474b-8a5a-8044c8617524 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:11.206334 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf2e772-5667-474b-8a5a-8044c8617524 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:11.206620 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf2e772-5667-474b-8a5a-8044c8617524 None admin] PATH_INFO: `/v3/projects/961f8057321f482d9a5d9202c7bc3c4c` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:11.209861 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf2e772-5667-474b-8a5a-8044c8617524 None admin] RBAC: Authorizing `identity:get_project(project_id=961f8057321f482d9a5d9202c7bc3c4c)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:11.213681 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf2e772-5667-474b-8a5a-8044c8617524 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:11.218432 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 132/263] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:29:10 2022] GET /identity/v3/projects/961f8057321f482d9a5d9202c7bc3c4c => generated 302 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:11.219224 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf2e772-5667-474b-8a5a-8044c8617524 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:11.272014 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:11.352819 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5119581c-d050-4b3f-a2b2-97f62643e4bf None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:11.408238 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf2e772-5667-474b-8a5a-8044c8617524 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:11.422084 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac6d28f-5735-4adc-a8fd-0fac64769614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:11.422695 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac6d28f-5735-4adc-a8fd-0fac64769614 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:11.422968 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac6d28f-5735-4adc-a8fd-0fac64769614 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:11.423250 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac6d28f-5735-4adc-a8fd-0fac64769614 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:11.471171 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32255175-b642-4e68-80a5-6d0d2b23c932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:11.471837 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32255175-b642-4e68-80a5-6d0d2b23c932 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:11.472114 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32255175-b642-4e68-80a5-6d0d2b23c932 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:11.472437 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32255175-b642-4e68-80a5-6d0d2b23c932 None admin] PATH_INFO: `/v3/projects/961f8057321f482d9a5d9202c7bc3c4c` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:11.473674 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32255175-b642-4e68-80a5-6d0d2b23c932 None admin] RBAC: Authorizing `identity:get_project(project_id=961f8057321f482d9a5d9202c7bc3c4c)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:11.475116 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32255175-b642-4e68-80a5-6d0d2b23c932 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:11.476778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 133/264] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:29:11 2022] GET /identity/v3/projects/961f8057321f482d9a5d9202c7bc3c4c => generated 302 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:11.477511 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32255175-b642-4e68-80a5-6d0d2b23c932 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:11.543152 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac6d28f-5735-4adc-a8fd-0fac64769614 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:11.545055 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac6d28f-5735-4adc-a8fd-0fac64769614 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:11.578002 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:11.643565 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:11.654346 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32255175-b642-4e68-80a5-6d0d2b23c932 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:11.715972 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 132/265] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:29:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:29:11.716866 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac6d28f-5735-4adc-a8fd-0fac64769614 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:11.724671 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ad9410-53d5-4afb-ac5e-7ad39fb4275b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:11.725351 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad9410-53d5-4afb-ac5e-7ad39fb4275b None admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:11.725633 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad9410-53d5-4afb-ac5e-7ad39fb4275b None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:11.725910 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad9410-53d5-4afb-ac5e-7ad39fb4275b None admin] PATH_INFO: `/v3/projects/961f8057321f482d9a5d9202c7bc3c4c/users/10ea7abdff734626b2ffe657de881f08/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:11.728564 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ad9410-53d5-4afb-ac5e-7ad39fb4275b None admin] RBAC: Authorizing `identity:create_grant(project_id=961f8057321f482d9a5d9202c7bc3c4c, user_id=10ea7abdff734626b2ffe657de881f08, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:11.730129 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ad9410-53d5-4afb-ac5e-7ad39fb4275b None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:11.748045 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 134/266] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 30 14:29:11 2022] PUT /identity/v3/projects/961f8057321f482d9a5d9202c7bc3c4c/users/10ea7abdff734626b2ffe657de881f08/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:29:11.748893 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ad9410-53d5-4afb-ac5e-7ad39fb4275b None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:11.840557 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac6d28f-5735-4adc-a8fd-0fac64769614 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:11.876185 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ad9410-53d5-4afb-ac5e-7ad39fb4275b None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:11.880289 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eb45f4-9096-4ab1-aa8c-90a2e49913e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:11.880918 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb45f4-9096-4ab1-aa8c-90a2e49913e0 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:11.881197 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb45f4-9096-4ab1-aa8c-90a2e49913e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:11.881465 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb45f4-9096-4ab1-aa8c-90a2e49913e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:11.882209 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eb45f4-9096-4ab1-aa8c-90a2e49913e0 None admin] RBAC: Adding query filter params (role.id=3cda0615d87d4af0860a2a5e9b4e2c05, scope.project.id=961f8057321f482d9a5d9202c7bc3c4c, user.id=05d150e8f6f049bd906ecda0491014a6) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:11.882584 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eb45f4-9096-4ab1-aa8c-90a2e49913e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:11.884685 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eb45f4-9096-4ab1-aa8c-90a2e49913e0 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:11.893618 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 133/267] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:29:11 2022] GET /identity/v3/role_assignments?user.id=05d150e8f6f049bd906ecda0491014a6&scope.project.id=961f8057321f482d9a5d9202c7bc3c4c&role.id=3cda0615d87d4af0860a2a5e9b4e2c05 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:11.894195 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:11.895027 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45772483-0c05-4c17-91ae-fa9239b26a5a None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:11.895360 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45772483-0c05-4c17-91ae-fa9239b26a5a None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:11.895639 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45772483-0c05-4c17-91ae-fa9239b26a5a None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:11.902415 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 134/268] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:11.904923 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c9a20-5fce-4642-918b-3cf1ce55e769 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:11.905214 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c9a20-5fce-4642-918b-3cf1ce55e769 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:11.905491 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c9a20-5fce-4642-918b-3cf1ce55e769 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:11.907588 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 135/269] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:11.908814 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed41aeb-7b30-42d1-9409-76894bda2175 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:11.909095 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed41aeb-7b30-42d1-9409-76894bda2175 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:11.909364 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed41aeb-7b30-42d1-9409-76894bda2175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:11.930943 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046591f1-c5c9-49fd-9097-12ddf97bae9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:11.931629 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046591f1-c5c9-49fd-9097-12ddf97bae9e None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:11.931906 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046591f1-c5c9-49fd-9097-12ddf97bae9e None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:11.932176 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046591f1-c5c9-49fd-9097-12ddf97bae9e None admin] PATH_INFO: `/v3/projects/961f8057321f482d9a5d9202c7bc3c4c` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:11.933730 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046591f1-c5c9-49fd-9097-12ddf97bae9e None admin] RBAC: Authorizing `identity:get_project(project_id=961f8057321f482d9a5d9202c7bc3c4c)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:11.936227 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046591f1-c5c9-49fd-9097-12ddf97bae9e None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:11.938268 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 135/270] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:29:11 2022] GET /identity/v3/projects/961f8057321f482d9a5d9202c7bc3c4c => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:11.943354 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479231b1-c22b-4a59-9f2c-ffe1225a130e None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:11.943354 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479231b1-c22b-4a59-9f2c-ffe1225a130e None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:11.943354 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479231b1-c22b-4a59-9f2c-ffe1225a130e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:11.944946 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-aed41aeb-7b30-42d1-9409-76894bda2175 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:11.946631 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-aed41aeb-7b30-42d1-9409-76894bda2175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:11.964035 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-479231b1-c22b-4a59-9f2c-ffe1225a130e None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:11.965255 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-479231b1-c22b-4a59-9f2c-ffe1225a130e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:11.968558 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:12.048809 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 136/271] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:12.050755 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed41aeb-7b30-42d1-9409-76894bda2175 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:12.059963 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 136/272] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:12.179566 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed41aeb-7b30-42d1-9409-76894bda2175 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:12.241190 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50b9bed-2e80-49bd-bdd1-ff74f8748987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:12.241729 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b9bed-2e80-49bd-bdd1-ff74f8748987 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:12.242012 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b9bed-2e80-49bd-bdd1-ff74f8748987 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:12.242284 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b9bed-2e80-49bd-bdd1-ff74f8748987 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:12.243032 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50b9bed-2e80-49bd-bdd1-ff74f8748987 None admin] RBAC: Adding query filter params (role.id=1c4886a7a7fd402d8221a0158ddcb167, scope.project.id=961f8057321f482d9a5d9202c7bc3c4c, user.id=05d150e8f6f049bd906ecda0491014a6) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:12.243367 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50b9bed-2e80-49bd-bdd1-ff74f8748987 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:12.244802 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50b9bed-2e80-49bd-bdd1-ff74f8748987 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:12.258257 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 137/273] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:29:12 2022] GET /identity/v3/role_assignments?user.id=05d150e8f6f049bd906ecda0491014a6&scope.project.id=961f8057321f482d9a5d9202c7bc3c4c&role.id=1c4886a7a7fd402d8221a0158ddcb167 => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:12.375000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479231b1-c22b-4a59-9f2c-ffe1225a130e None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:12.482138 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50b9bed-2e80-49bd-bdd1-ff74f8748987 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:12.503777 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:12.504234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479231b1-c22b-4a59-9f2c-ffe1225a130e None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:12.519378 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:12.552461 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5de1149-0a1b-4065-b7a1-f098538d684f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:12.553113 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de1149-0a1b-4065-b7a1-f098538d684f None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:12.553941 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de1149-0a1b-4065-b7a1-f098538d684f None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:12.554266 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de1149-0a1b-4065-b7a1-f098538d684f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:12.569530 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5de1149-0a1b-4065-b7a1-f098538d684f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:12.575523 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-f5de1149-0a1b-4065-b7a1-f098538d684f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:29:12.575523 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:12.575523 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:12.575523 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:12.575523 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:12.575523 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:12.575523 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:12.575523 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:12.575523 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:12.575523 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:12.575523 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:12.575523 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:12.575523 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:12.575523 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:12.575523 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:12.575523 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:12.576416 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:12.576416 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:12.576416 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:12.576416 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:12.576416 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:12.576416 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:12.576416 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:12.576416 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:12.576416 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:12.576416 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:12.576416 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:12.576416 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:12.576416 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:12.576416 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:12.576416 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:12.576416 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:12.577234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:12.577234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:12.577234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:12.577234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:12.577234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:12.577234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:12.577234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:12.577234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:12.577234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:12.577234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:12.577234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:29:12.577234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:12.577234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 137/274] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:29:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:12.588221 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5de1149-0a1b-4065-b7a1-f098538d684f None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:12.609394 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50b9bed-2e80-49bd-bdd1-ff74f8748987 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:12.653579 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cdd793-0ad0-48ba-8418-a65fe8c27202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:12.654242 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cdd793-0ad0-48ba-8418-a65fe8c27202 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:12.654589 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cdd793-0ad0-48ba-8418-a65fe8c27202 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:12.654864 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cdd793-0ad0-48ba-8418-a65fe8c27202 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:12.656036 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cdd793-0ad0-48ba-8418-a65fe8c27202 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:12.657485 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cdd793-0ad0-48ba-8418-a65fe8c27202 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:12.664259 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 138/275] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:29:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:12.666951 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cdd793-0ad0-48ba-8418-a65fe8c27202 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:12.733863 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5de1149-0a1b-4065-b7a1-f098538d684f None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:12.766899 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:12.780260 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d900ae4-6ae8-47ce-b663-402087ee8353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:12.780954 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d900ae4-6ae8-47ce-b663-402087ee8353 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:12.781306 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d900ae4-6ae8-47ce-b663-402087ee8353 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:12.781649 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d900ae4-6ae8-47ce-b663-402087ee8353 None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:12.782377 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d900ae4-6ae8-47ce-b663-402087ee8353 None admin] RBAC: Adding query filter params (name=member) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:12.782920 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d900ae4-6ae8-47ce-b663-402087ee8353 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:12.784255 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d900ae4-6ae8-47ce-b663-402087ee8353 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:12.789440 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cdd793-0ad0-48ba-8418-a65fe8c27202 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:12.792611 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 138/276] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 30 14:29:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:12.802236 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d900ae4-6ae8-47ce-b663-402087ee8353 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:12.827491 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0816d1-5bac-43f6-8832-c950f1d47a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:12.828105 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0816d1-5bac-43f6-8832-c950f1d47a1a None admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:12.828397 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0816d1-5bac-43f6-8832-c950f1d47a1a None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:12.828675 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0816d1-5bac-43f6-8832-c950f1d47a1a None admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:12.829319 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0816d1-5bac-43f6-8832-c950f1d47a1a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:12.830997 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0816d1-5bac-43f6-8832-c950f1d47a1a None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:12.834063 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:12.867397 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 139/277] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 30 14:29:12 2022] POST /identity/v3/users => generated 293 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:29:12.927398 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d900ae4-6ae8-47ce-b663-402087ee8353 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:12.968745 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32176ca8-2d16-4070-beff-d27f1fecb081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:12.969651 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32176ca8-2d16-4070-beff-d27f1fecb081 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:12.970105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32176ca8-2d16-4070-beff-d27f1fecb081 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:12.970544 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32176ca8-2d16-4070-beff-d27f1fecb081 None admin] PATH_INFO: `/v3/users/e4b454e73b9d437e969b865e5f7f23a1` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:12.986807 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32176ca8-2d16-4070-beff-d27f1fecb081 None admin] RBAC: Authorizing `identity:get_user(user_id=e4b454e73b9d437e969b865e5f7f23a1)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:12.988871 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32176ca8-2d16-4070-beff-d27f1fecb081 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:12.991221 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 139/278] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:29:12 2022] GET /identity/v3/users/e4b454e73b9d437e969b865e5f7f23a1 => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:12.998308 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0816d1-5bac-43f6-8832-c950f1d47a1a None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:13.090801 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94502992-0cc2-49f9-b811-0775a5ed3ed4 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:13.091269 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94502992-0cc2-49f9-b811-0775a5ed3ed4 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:13.091570 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94502992-0cc2-49f9-b811-0775a5ed3ed4 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:13.093000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 140/279] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:29:13.098738 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ca71f-3b39-481b-8c01-dc9667c6a5c0 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:13.099083 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ca71f-3b39-481b-8c01-dc9667c6a5c0 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:13.099317 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ca71f-3b39-481b-8c01-dc9667c6a5c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:13.111496 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0816d1-5bac-43f6-8832-c950f1d47a1a None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:13.119496 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-1c5ca71f-3b39-481b-8c01-dc9667c6a5c0 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:13.120691 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-1c5ca71f-3b39-481b-8c01-dc9667c6a5c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:13.164018 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c97151-cd73-4e48-88cc-8f758e6dd232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:13.164729 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c97151-cd73-4e48-88cc-8f758e6dd232 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:13.165075 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c97151-cd73-4e48-88cc-8f758e6dd232 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:13.165418 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c97151-cd73-4e48-88cc-8f758e6dd232 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:13.166085 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c97151-cd73-4e48-88cc-8f758e6dd232 None admin] RBAC: Adding query filter params (role.id=6a985d5659a3416cba33ca9d30b39f85, user.id=e4b454e73b9d437e969b865e5f7f23a1, scope.system=all) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:13.166394 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c97151-cd73-4e48-88cc-8f758e6dd232 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:13.168031 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c97151-cd73-4e48-88cc-8f758e6dd232 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:13.174796 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 140/280] 10.222.0.30 () {56 vars in 1308 bytes} [Tue Aug 30 14:29:12 2022] GET /identity/v3/role_assignments?user.id=e4b454e73b9d437e969b865e5f7f23a1&scope.system=all&role.id=6a985d5659a3416cba33ca9d30b39f85 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:13.218966 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 141/281] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:13.501774 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c97151-cd73-4e48-88cc-8f758e6dd232 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:13.504712 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:13.629269 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c97151-cd73-4e48-88cc-8f758e6dd232 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:13.674924 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2045c1-42b8-45ea-9c40-057df13c0589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:13.675938 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2045c1-42b8-45ea-9c40-057df13c0589 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:13.676426 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2045c1-42b8-45ea-9c40-057df13c0589 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:13.676887 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2045c1-42b8-45ea-9c40-057df13c0589 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:13.692307 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2045c1-42b8-45ea-9c40-057df13c0589 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:13.697902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-ef2045c1-42b8-45ea-9c40-057df13c0589 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:29:13.697902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:13.697902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:13.697902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:13.697902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:13.697902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:13.697902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:13.697902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:13.697902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:13.697902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:13.697902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:13.697902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:13.697902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:13.697902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:13.697902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:13.697902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:13.698779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:13.698779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:13.698779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:13.698779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:13.698779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:13.698779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:13.698779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:13.698779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:13.698779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:13.698779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:13.698779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:13.698779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:13.698779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:13.698779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:13.698779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:13.698779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:13.699712 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:13.699712 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:13.699712 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:13.699712 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:13.699712 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:13.699712 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:13.699712 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:13.699712 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:13.699712 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:13.699712 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:13.699712 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:29:13.699712 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:13.702166 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 141/282] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:29:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:13.708103 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5ca71f-3b39-481b-8c01-dc9667c6a5c0 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:13.745537 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:13.863009 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5ca71f-3b39-481b-8c01-dc9667c6a5c0 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:13.913309 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a2ff64-509a-4335-8124-7ae0092215bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:13.913934 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2ff64-509a-4335-8124-7ae0092215bc None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:13.914280 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2ff64-509a-4335-8124-7ae0092215bc None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:13.914555 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2ff64-509a-4335-8124-7ae0092215bc None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:13.915305 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a2ff64-509a-4335-8124-7ae0092215bc None admin] RBAC: Adding query filter params (name=reader) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:13.915614 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a2ff64-509a-4335-8124-7ae0092215bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:13.916990 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a2ff64-509a-4335-8124-7ae0092215bc None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:13.927786 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 142/283] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 30 14:29:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:13.933201 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2045c1-42b8-45ea-9c40-057df13c0589 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:13.997861 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:14.049738 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2045c1-42b8-45ea-9c40-057df13c0589 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:14.091160 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8405cf4b-f672-4069-9a6d-e12bd57dc0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:14.091830 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8405cf4b-f672-4069-9a6d-e12bd57dc0f5 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:14.092115 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8405cf4b-f672-4069-9a6d-e12bd57dc0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:14.092392 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8405cf4b-f672-4069-9a6d-e12bd57dc0f5 None admin] PATH_INFO: `/v3/users/aa171c8644ee4d23babe25ebdaa1cf70` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:14.093550 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8405cf4b-f672-4069-9a6d-e12bd57dc0f5 None admin] RBAC: Authorizing `identity:get_user(user_id=aa171c8644ee4d23babe25ebdaa1cf70)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:14.094992 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8405cf4b-f672-4069-9a6d-e12bd57dc0f5 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:14.096693 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 142/284] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:29:13 2022] GET /identity/v3/users/aa171c8644ee4d23babe25ebdaa1cf70 => generated 297 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:14.102359 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a2ff64-509a-4335-8124-7ae0092215bc None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:14.216805 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:14.224976 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae629fb-8ed1-4b1f-896b-abac5595f0bf None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:14.225239 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae629fb-8ed1-4b1f-896b-abac5595f0bf None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:14.225500 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae629fb-8ed1-4b1f-896b-abac5595f0bf None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:14.226853 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 143/285] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:29:14.231968 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c494706-c451-46e4-9492-a05d8f1f1b25 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:14.232256 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c494706-c451-46e4-9492-a05d8f1f1b25 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:14.232534 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c494706-c451-46e4-9492-a05d8f1f1b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:14.243866 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a2ff64-509a-4335-8124-7ae0092215bc None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:14.259577 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-4c494706-c451-46e4-9492-a05d8f1f1b25 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:14.260777 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-4c494706-c451-46e4-9492-a05d8f1f1b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:14.303980 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4532bbca-8361-485c-9413-eaae6976b1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:14.304626 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4532bbca-8361-485c-9413-eaae6976b1e8 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:14.304906 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4532bbca-8361-485c-9413-eaae6976b1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:14.305181 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4532bbca-8361-485c-9413-eaae6976b1e8 None admin] PATH_INFO: `/v3/projects/961f8057321f482d9a5d9202c7bc3c4c` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:14.306328 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4532bbca-8361-485c-9413-eaae6976b1e8 None admin] RBAC: Authorizing `identity:get_project(project_id=961f8057321f482d9a5d9202c7bc3c4c)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:14.307980 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4532bbca-8361-485c-9413-eaae6976b1e8 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:14.310254 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 143/286] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:29:14 2022] GET /identity/v3/projects/961f8057321f482d9a5d9202c7bc3c4c => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:14.316596 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4532bbca-8361-485c-9413-eaae6976b1e8 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:14.416479 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 144/287] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:29:14.460545 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4532bbca-8361-485c-9413-eaae6976b1e8 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:14.495111 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74742482-287b-405e-954d-3686e5f98a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:14.496141 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74742482-287b-405e-954d-3686e5f98a95 None admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:14.497209 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74742482-287b-405e-954d-3686e5f98a95 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:14.497671 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74742482-287b-405e-954d-3686e5f98a95 None admin] PATH_INFO: `/v3/projects/961f8057321f482d9a5d9202c7bc3c4c/users/aa171c8644ee4d23babe25ebdaa1cf70/roles/f51dfe78039041e3a947ea27cfe97651` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:14.499805 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74742482-287b-405e-954d-3686e5f98a95 None admin] RBAC: Authorizing `identity:create_grant(project_id=961f8057321f482d9a5d9202c7bc3c4c, user_id=aa171c8644ee4d23babe25ebdaa1cf70, role_id=f51dfe78039041e3a947ea27cfe97651)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:14.501523 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74742482-287b-405e-954d-3686e5f98a95 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:14.516886 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 144/288] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 30 14:29:14 2022] PUT /identity/v3/projects/961f8057321f482d9a5d9202c7bc3c4c/users/aa171c8644ee4d23babe25ebdaa1cf70/roles/f51dfe78039041e3a947ea27cfe97651 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:29:14.776174 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c494706-c451-46e4-9492-a05d8f1f1b25 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:14.840049 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:14.883406 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c494706-c451-46e4-9492-a05d8f1f1b25 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:14.917986 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d4ac9-76c8-485c-8d88-ebf611e60203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:14.919298 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d4ac9-76c8-485c-8d88-ebf611e60203 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:14.919582 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d4ac9-76c8-485c-8d88-ebf611e60203 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:14.919856 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d4ac9-76c8-485c-8d88-ebf611e60203 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:14.930776 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884d4ac9-76c8-485c-8d88-ebf611e60203 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:14.936760 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-884d4ac9-76c8-485c-8d88-ebf611e60203 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:29:14.936760 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:14.936760 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:14.936760 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:14.936760 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:14.936760 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:14.936760 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:14.936760 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:14.936760 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:14.936760 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:14.936760 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:14.936760 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:14.936760 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:14.936760 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:14.936760 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:14.936760 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:14.937636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:14.937636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:14.937636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:14.937636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:14.937636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:14.937636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:14.937636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:14.937636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:14.937636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:14.937636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:14.937636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:14.937636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:14.937636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:14.937636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:14.937636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:14.937636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:14.938421 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:14.938421 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:14.938421 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:14.938421 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:14.938421 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:14.938421 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:14.938421 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:14.938421 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:14.938421 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:14.938421 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:14.938421 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:29:14.938421 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:14.941042 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 145/289] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:29:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:14.945203 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74742482-287b-405e-954d-3686e5f98a95 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:14.974316 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:15.074467 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74742482-287b-405e-954d-3686e5f98a95 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:15.110511 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb0e06-3e8f-4904-bbee-d6ceb7be1398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:15.111476 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb0e06-3e8f-4904-bbee-d6ceb7be1398 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:15.111896 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb0e06-3e8f-4904-bbee-d6ceb7be1398 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:15.112349 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb0e06-3e8f-4904-bbee-d6ceb7be1398 None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:15.113146 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eb0e06-3e8f-4904-bbee-d6ceb7be1398 None admin] RBAC: Adding query filter params (name=member) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:15.113582 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eb0e06-3e8f-4904-bbee-d6ceb7be1398 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:15.115102 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eb0e06-3e8f-4904-bbee-d6ceb7be1398 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:15.124483 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 145/290] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 30 14:29:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:15.136312 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d4ac9-76c8-485c-8d88-ebf611e60203 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:15.257252 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d4ac9-76c8-485c-8d88-ebf611e60203 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:15.278879 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:15.293085 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a2c635-d4de-4d0b-bb77-6b7cb0b7f4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:15.293769 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2c635-d4de-4d0b-bb77-6b7cb0b7f4e0 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:15.294074 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2c635-d4de-4d0b-bb77-6b7cb0b7f4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:15.294342 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2c635-d4de-4d0b-bb77-6b7cb0b7f4e0 None admin] PATH_INFO: `/v3/users/10ea7abdff734626b2ffe657de881f08` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:15.295565 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a2c635-d4de-4d0b-bb77-6b7cb0b7f4e0 None admin] RBAC: Authorizing `identity:get_user(user_id=10ea7abdff734626b2ffe657de881f08)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:15.297022 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a2c635-d4de-4d0b-bb77-6b7cb0b7f4e0 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:15.298628 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 146/291] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:29:15 2022] GET /identity/v3/users/10ea7abdff734626b2ffe657de881f08 => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:15.304649 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb0e06-3e8f-4904-bbee-d6ceb7be1398 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:15.392628 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:15.394776 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74739df1-457e-4772-a931-c30679641721 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:15.395319 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74739df1-457e-4772-a931-c30679641721 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:15.395826 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74739df1-457e-4772-a931-c30679641721 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:15.397654 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 147/292] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:15.406120 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8108ecaa-9d27-40d8-beb1-be90d971af74 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:15.406120 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8108ecaa-9d27-40d8-beb1-be90d971af74 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:15.406120 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8108ecaa-9d27-40d8-beb1-be90d971af74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:15.432857 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-8108ecaa-9d27-40d8-beb1-be90d971af74 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:15.435131 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-8108ecaa-9d27-40d8-beb1-be90d971af74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:15.455247 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb0e06-3e8f-4904-bbee-d6ceb7be1398 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:15.515810 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe5c45-3d60-485f-b6c9-93f5210dcac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:15.516437 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe5c45-3d60-485f-b6c9-93f5210dcac6 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:15.516722 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe5c45-3d60-485f-b6c9-93f5210dcac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:15.516998 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe5c45-3d60-485f-b6c9-93f5210dcac6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:15.545689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 148/293] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:15.594097 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ba589-46dd-4993-91d1-c4c97408b5cd None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:15.594424 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ba589-46dd-4993-91d1-c4c97408b5cd None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:15.594652 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ba589-46dd-4993-91d1-c4c97408b5cd None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:15.596251 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 149/294] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:29:15.601096 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028b953-efe6-487a-bdac-8d5ba049892d None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:15.601379 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028b953-efe6-487a-bdac-8d5ba049892d None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:15.601657 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028b953-efe6-487a-bdac-8d5ba049892d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:15.609461 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe5c45-3d60-485f-b6c9-93f5210dcac6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:15.611971 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe5c45-3d60-485f-b6c9-93f5210dcac6 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:15.628170 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-4028b953-efe6-487a-bdac-8d5ba049892d None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:15.629369 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-4028b953-efe6-487a-bdac-8d5ba049892d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:15.727661 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 150/295] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:29:15.729305 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cbab0e-f210-4e74-94cb-782f5d29e06d None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:15.729594 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cbab0e-f210-4e74-94cb-782f5d29e06d None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:15.729866 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cbab0e-f210-4e74-94cb-782f5d29e06d None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:15.731883 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 151/296] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:15.737197 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd7311-ed2f-46a8-b718-c095539b46a3 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:15.737656 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd7311-ed2f-46a8-b718-c095539b46a3 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:15.737937 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd7311-ed2f-46a8-b718-c095539b46a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:15.738613 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:15.767342 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-fecd7311-ed2f-46a8-b718-c095539b46a3 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:15.768549 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-fecd7311-ed2f-46a8-b718-c095539b46a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:15.804550 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 146/297] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:29:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:29:15.811339 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe5c45-3d60-485f-b6c9-93f5210dcac6 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:15.855177 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:15.883828 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 152/298] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:15.884693 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecd7311-ed2f-46a8-b718-c095539b46a3 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:15.978952 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe5c45-3d60-485f-b6c9-93f5210dcac6 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:15.992804 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecd7311-ed2f-46a8-b718-c095539b46a3 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:16.032342 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2e400e-7955-4f48-b3c5-ca14c552fe25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:16.032983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e400e-7955-4f48-b3c5-ca14c552fe25 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:16.033266 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e400e-7955-4f48-b3c5-ca14c552fe25 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:16.033540 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e400e-7955-4f48-b3c5-ca14c552fe25 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:16.045383 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2e400e-7955-4f48-b3c5-ca14c552fe25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:16.050286 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-3a2e400e-7955-4f48-b3c5-ca14c552fe25 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:29:16.050286 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:16.050286 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:16.050286 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:16.050286 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:16.050286 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:16.050286 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:16.050286 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:16.050286 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:16.050286 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:16.050286 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:16.050286 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:16.050286 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:16.050286 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:16.050286 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:16.050286 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:16.051351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:16.051351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:16.051351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:16.051351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:16.051351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:16.051351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:16.051351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:16.051351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:16.051351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:16.051351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:16.051351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:16.051351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:16.051351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:16.051351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:16.051351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:16.051351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:16.052333 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:16.052333 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:16.052333 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:16.052333 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:16.052333 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:16.052333 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:16.052333 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:16.052333 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:16.052333 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:16.052333 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:16.052333 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:29:16.052333 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:16.052333 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 153/299] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:29:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:16.060254 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2e400e-7955-4f48-b3c5-ca14c552fe25 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:16.075374 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f93ade-9901-4ba7-a587-6570eecaa72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:16.076109 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f93ade-9901-4ba7-a587-6570eecaa72c None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:16.076389 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f93ade-9901-4ba7-a587-6570eecaa72c None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:16.076663 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f93ade-9901-4ba7-a587-6570eecaa72c None admin] PATH_INFO: `/v3/projects/961f8057321f482d9a5d9202c7bc3c4c` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:16.079793 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f93ade-9901-4ba7-a587-6570eecaa72c None admin] RBAC: Authorizing `identity:get_project(project_id=961f8057321f482d9a5d9202c7bc3c4c)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:16.081288 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f93ade-9901-4ba7-a587-6570eecaa72c None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:16.083053 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 147/300] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:29:15 2022] GET /identity/v3/projects/961f8057321f482d9a5d9202c7bc3c4c => generated 302 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:29:16.089922 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f93ade-9901-4ba7-a587-6570eecaa72c None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:16.134884 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:16.162447 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:16.206900 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2e400e-7955-4f48-b3c5-ca14c552fe25 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:16.231879 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f93ade-9901-4ba7-a587-6570eecaa72c None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:16.249489 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aeb0a0-041c-4107-ac7e-977f902b043e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:16.250103 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aeb0a0-041c-4107-ac7e-977f902b043e None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:16.250382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aeb0a0-041c-4107-ac7e-977f902b043e None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:16.250653 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aeb0a0-041c-4107-ac7e-977f902b043e None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:16.251343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aeb0a0-041c-4107-ac7e-977f902b043e None admin] RBAC: Adding query filter params (name=member) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:16.251645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aeb0a0-041c-4107-ac7e-977f902b043e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:16.253020 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aeb0a0-041c-4107-ac7e-977f902b043e None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:16.261750 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 154/301] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 30 14:29:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:16.267380 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aeb0a0-041c-4107-ac7e-977f902b043e None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:16.267525 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7761bd98-b93e-4775-9b3d-a720a3b89090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:16.268080 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7761bd98-b93e-4775-9b3d-a720a3b89090 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:16.268361 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7761bd98-b93e-4775-9b3d-a720a3b89090 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:16.268631 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7761bd98-b93e-4775-9b3d-a720a3b89090 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:16.269348 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7761bd98-b93e-4775-9b3d-a720a3b89090 None admin] RBAC: Adding query filter params (role.id=6a985d5659a3416cba33ca9d30b39f85, scope.project.id=961f8057321f482d9a5d9202c7bc3c4c, user.id=10ea7abdff734626b2ffe657de881f08) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:16.269649 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7761bd98-b93e-4775-9b3d-a720a3b89090 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:16.271928 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7761bd98-b93e-4775-9b3d-a720a3b89090 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:16.287239 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 148/302] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:29:16 2022] GET /identity/v3/role_assignments?user.id=10ea7abdff734626b2ffe657de881f08&scope.project.id=961f8057321f482d9a5d9202c7bc3c4c&role.id=6a985d5659a3416cba33ca9d30b39f85 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:16.287239 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7761bd98-b93e-4775-9b3d-a720a3b89090 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:16.411918 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7761bd98-b93e-4775-9b3d-a720a3b89090 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:16.415948 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:16.427108 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:16.430909 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aeb0a0-041c-4107-ac7e-977f902b043e None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:16.458191 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7644d0be-c9ab-4897-a732-c407c0521d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:16.459078 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644d0be-c9ab-4897-a732-c407c0521d22 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:16.459611 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644d0be-c9ab-4897-a732-c407c0521d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:16.460033 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644d0be-c9ab-4897-a732-c407c0521d22 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:16.461371 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7644d0be-c9ab-4897-a732-c407c0521d22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:16.462935 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7644d0be-c9ab-4897-a732-c407c0521d22 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:16.464648 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 149/303] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:29:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:16.466275 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7644d0be-c9ab-4897-a732-c407c0521d22 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:16.473767 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f21c9-2d9f-431b-a591-2f76c1394ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:16.474505 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f21c9-2d9f-431b-a591-2f76c1394ec1 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:16.474858 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f21c9-2d9f-431b-a591-2f76c1394ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:16.475202 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f21c9-2d9f-431b-a591-2f76c1394ec1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:16.488825 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2f21c9-2d9f-431b-a591-2f76c1394ec1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:16.495838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-1d2f21c9-2d9f-431b-a591-2f76c1394ec1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:29:16.495838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:16.495838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:16.495838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:16.495838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:16.495838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:16.495838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:16.495838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:16.495838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:16.495838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:16.495838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:16.495838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:16.495838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:16.495838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:16.495838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:16.495838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:16.497155 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:16.497155 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:16.497155 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:16.497155 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:16.497155 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:16.497155 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:16.497155 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:16.497155 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:16.497155 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:16.497155 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:16.497155 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:16.497155 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:16.497155 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:16.497155 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:16.497155 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:16.497155 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:16.497984 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:16.497984 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:16.497984 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:16.497984 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:16.497984 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:16.497984 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:16.497984 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:16.497984 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:16.497984 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:16.497984 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:16.497984 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:29:16.497984 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:16.503235 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 155/304] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:29:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:16.503235 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f21c9-2d9f-431b-a591-2f76c1394ec1 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:16.591819 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7644d0be-c9ab-4897-a732-c407c0521d22 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:16.620890 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f21c9-2d9f-431b-a591-2f76c1394ec1 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:16.628291 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf7793-0d77-4bb7-8f14-5b186535f8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:16.629262 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf7793-0d77-4bb7-8f14-5b186535f8f5 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:16.629723 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf7793-0d77-4bb7-8f14-5b186535f8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:16.630185 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf7793-0d77-4bb7-8f14-5b186535f8f5 None admin] PATH_INFO: `/v3/users/e4b454e73b9d437e969b865e5f7f23a1` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:16.631570 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbf7793-0d77-4bb7-8f14-5b186535f8f5 None admin] RBAC: Authorizing `identity:get_user(user_id=e4b454e73b9d437e969b865e5f7f23a1)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:16.633252 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbf7793-0d77-4bb7-8f14-5b186535f8f5 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:16.640580 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 150/305] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:29:16 2022] GET /identity/v3/users/e4b454e73b9d437e969b865e5f7f23a1 => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:16.641639 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf7793-0d77-4bb7-8f14-5b186535f8f5 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:16.656360 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad29eef-3de4-40d1-b19b-c2585f4b2ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:16.657245 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad29eef-3de4-40d1-b19b-c2585f4b2ec6 None admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:16.657698 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad29eef-3de4-40d1-b19b-c2585f4b2ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:16.658144 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad29eef-3de4-40d1-b19b-c2585f4b2ec6 None admin] PATH_INFO: `/v3/groups` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:16.658995 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad29eef-3de4-40d1-b19b-c2585f4b2ec6 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:16.660651 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad29eef-3de4-40d1-b19b-c2585f4b2ec6 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:16.677329 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 156/306] 10.222.0.30 () {60 vars in 1134 bytes} [Tue Aug 30 14:29:16 2022] POST /identity/v3/groups => generated 229 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:29:16.678375 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad29eef-3de4-40d1-b19b-c2585f4b2ec6 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:16.698707 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:16.709097 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:16.796751 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf7793-0d77-4bb7-8f14-5b186535f8f5 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:16.821811 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad29eef-3de4-40d1-b19b-c2585f4b2ec6 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:16.839492 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b627a-6ae6-4dfd-9cf4-6f88a7a0d281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:16.840592 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b627a-6ae6-4dfd-9cf4-6f88a7a0d281 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:16.841055 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b627a-6ae6-4dfd-9cf4-6f88a7a0d281 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:16.841534 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b627a-6ae6-4dfd-9cf4-6f88a7a0d281 None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:16.842432 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90b627a-6ae6-4dfd-9cf4-6f88a7a0d281 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:16.842744 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90b627a-6ae6-4dfd-9cf4-6f88a7a0d281 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:16.844148 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90b627a-6ae6-4dfd-9cf4-6f88a7a0d281 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:16.854524 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 151/307] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 30 14:29:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:16.859651 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b627a-6ae6-4dfd-9cf4-6f88a7a0d281 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:16.867518 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0218b01a-7998-4495-8f5d-fe1969b462ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:16.868186 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0218b01a-7998-4495-8f5d-fe1969b462ec None admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:16.868884 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0218b01a-7998-4495-8f5d-fe1969b462ec None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:16.869166 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0218b01a-7998-4495-8f5d-fe1969b462ec None admin] PATH_INFO: `/v3/system/users/e4b454e73b9d437e969b865e5f7f23a1/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:16.870997 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0218b01a-7998-4495-8f5d-fe1969b462ec None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e4b454e73b9d437e969b865e5f7f23a1, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:16.874108 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0218b01a-7998-4495-8f5d-fe1969b462ec None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:16.891177 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 157/308] 10.222.0.30 () {58 vars in 1334 bytes} [Tue Aug 30 14:29:16 2022] PUT /identity/v3/system/users/e4b454e73b9d437e969b865e5f7f23a1/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:29:16.935730 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c82ff8b-2751-4a16-8bf1-9a997168b384 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:16.936045 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c82ff8b-2751-4a16-8bf1-9a997168b384 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:16.936288 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c82ff8b-2751-4a16-8bf1-9a997168b384 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:16.938001 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 158/309] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:29:16.943477 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f8e74d-9e25-4c64-ab34-fdbc8d01b1c2 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:16.943975 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f8e74d-9e25-4c64-ab34-fdbc8d01b1c2 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:16.944443 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f8e74d-9e25-4c64-ab34-fdbc8d01b1c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:16.964400 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-59f8e74d-9e25-4c64-ab34-fdbc8d01b1c2 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:16.966036 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-59f8e74d-9e25-4c64-ab34-fdbc8d01b1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:16.985831 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b627a-6ae6-4dfd-9cf4-6f88a7a0d281 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:16.987978 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:17.026238 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d944895-91c7-47fd-b180-2028a54fd183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:17.026917 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d944895-91c7-47fd-b180-2028a54fd183 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:17.027740 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d944895-91c7-47fd-b180-2028a54fd183 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:17.028028 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d944895-91c7-47fd-b180-2028a54fd183 None admin] PATH_INFO: `/v3/users/999b71fa2e67451ba9d3140167051c4f` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:17.047065 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d944895-91c7-47fd-b180-2028a54fd183 None admin] RBAC: Authorizing `identity:get_user(user_id=999b71fa2e67451ba9d3140167051c4f)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:17.048535 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d944895-91c7-47fd-b180-2028a54fd183 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:17.048934 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 159/310] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:29:17.053029 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 152/311] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:29:16 2022] GET /identity/v3/users/999b71fa2e67451ba9d3140167051c4f => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:29:17.058998 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f8e74d-9e25-4c64-ab34-fdbc8d01b1c2 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:17.079581 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:17.171010 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f8e74d-9e25-4c64-ab34-fdbc8d01b1c2 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:17.214682 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb9d6b6-c701-44cc-9fe0-97098d214e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:17.215621 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb9d6b6-c701-44cc-9fe0-97098d214e14 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:17.216103 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb9d6b6-c701-44cc-9fe0-97098d214e14 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:17.216570 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb9d6b6-c701-44cc-9fe0-97098d214e14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:17.217500 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb9d6b6-c701-44cc-9fe0-97098d214e14 None admin] RBAC: Adding query filter params (role.id=f51dfe78039041e3a947ea27cfe97651, user.id=999b71fa2e67451ba9d3140167051c4f, scope.system=all) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:17.218026 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb9d6b6-c701-44cc-9fe0-97098d214e14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:17.219732 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb9d6b6-c701-44cc-9fe0-97098d214e14 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:17.226067 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 160/312] 10.222.0.30 () {56 vars in 1308 bytes} [Tue Aug 30 14:29:17 2022] GET /identity/v3/role_assignments?user.id=999b71fa2e67451ba9d3140167051c4f&scope.system=all&role.id=f51dfe78039041e3a947ea27cfe97651 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:17.320020 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d944895-91c7-47fd-b180-2028a54fd183 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:17.429033 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d944895-91c7-47fd-b180-2028a54fd183 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:17.464648 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a14081-bf18-450d-be3d-674478d5394a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:17.465958 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a14081-bf18-450d-be3d-674478d5394a None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:17.466246 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a14081-bf18-450d-be3d-674478d5394a None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:17.466520 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a14081-bf18-450d-be3d-674478d5394a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:17.476942 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a14081-bf18-450d-be3d-674478d5394a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:17.482069 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-09a14081-bf18-450d-be3d-674478d5394a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:29:17.482069 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:17.482069 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:17.482069 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:17.482069 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:17.482069 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:17.482069 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:17.482069 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:17.482069 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:17.482069 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:17.482069 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:17.482069 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:17.482069 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:17.482069 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:17.482069 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:17.482069 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:17.482985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:17.482985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:17.482985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:17.482985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:17.482985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:17.482985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:17.482985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:17.482985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:17.482985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:17.482985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:17.482985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:17.482985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:17.482985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:17.482985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:17.482985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:17.482985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:17.483826 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:17.483826 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:17.483826 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:17.483826 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:17.483826 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:17.483826 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:17.483826 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:17.483826 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:17.483826 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:17.483826 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:17.483826 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:29:17.483826 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:17.483826 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 153/313] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:29:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:17.489102 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb9d6b6-c701-44cc-9fe0-97098d214e14 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:17.569355 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:17.592549 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb9d6b6-c701-44cc-9fe0-97098d214e14 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:17.626754 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df6fc71-c4c9-4f87-bfe3-1192e7938c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:17.627395 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df6fc71-c4c9-4f87-bfe3-1192e7938c54 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:17.627749 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df6fc71-c4c9-4f87-bfe3-1192e7938c54 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:17.628017 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df6fc71-c4c9-4f87-bfe3-1192e7938c54 None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:17.628681 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df6fc71-c4c9-4f87-bfe3-1192e7938c54 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:17.628979 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df6fc71-c4c9-4f87-bfe3-1192e7938c54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:17.630408 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df6fc71-c4c9-4f87-bfe3-1192e7938c54 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:17.639709 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 161/314] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 30 14:29:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:17.645249 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a14081-bf18-450d-be3d-674478d5394a None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:17.667906 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:17.753148 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a14081-bf18-450d-be3d-674478d5394a None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:17.789114 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bbfeb3-1c0a-4931-8a10-f2b8c57609ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:17.789991 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bbfeb3-1c0a-4931-8a10-f2b8c57609ca None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:17.790413 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bbfeb3-1c0a-4931-8a10-f2b8c57609ca None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:17.790829 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bbfeb3-1c0a-4931-8a10-f2b8c57609ca None admin] PATH_INFO: `/v3/users/aa171c8644ee4d23babe25ebdaa1cf70` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:17.792171 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bbfeb3-1c0a-4931-8a10-f2b8c57609ca None admin] RBAC: Authorizing `identity:get_user(user_id=aa171c8644ee4d23babe25ebdaa1cf70)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:17.794287 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bbfeb3-1c0a-4931-8a10-f2b8c57609ca None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:17.796367 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 154/315] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:29:17 2022] GET /identity/v3/users/aa171c8644ee4d23babe25ebdaa1cf70 => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:17.801840 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df6fc71-c4c9-4f87-bfe3-1192e7938c54 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:17.907634 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df6fc71-c4c9-4f87-bfe3-1192e7938c54 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:17.945806 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943fb14b-9099-48d8-bba2-676f3400ae8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:17.946415 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943fb14b-9099-48d8-bba2-676f3400ae8b None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:17.946697 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943fb14b-9099-48d8-bba2-676f3400ae8b None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:17.946968 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943fb14b-9099-48d8-bba2-676f3400ae8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:17.957760 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:18.017269 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943fb14b-9099-48d8-bba2-676f3400ae8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:18.018728 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943fb14b-9099-48d8-bba2-676f3400ae8b None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:18.121851 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 162/316] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:29:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:29:18.127196 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bbfeb3-1c0a-4931-8a10-f2b8c57609ca None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:18.214742 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:18.237010 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bbfeb3-1c0a-4931-8a10-f2b8c57609ca None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:18.287302 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ac6ab-bdae-47ee-b900-b0cfdaeb6ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:18.287948 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ac6ab-bdae-47ee-b900-b0cfdaeb6ca5 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:18.288224 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ac6ab-bdae-47ee-b900-b0cfdaeb6ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:18.288491 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ac6ab-bdae-47ee-b900-b0cfdaeb6ca5 None admin] PATH_INFO: `/v3/projects/961f8057321f482d9a5d9202c7bc3c4c` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:18.289625 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714ac6ab-bdae-47ee-b900-b0cfdaeb6ca5 None admin] RBAC: Authorizing `identity:get_project(project_id=961f8057321f482d9a5d9202c7bc3c4c)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:18.291053 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714ac6ab-bdae-47ee-b900-b0cfdaeb6ca5 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:18.293017 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 155/317] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:29:18 2022] GET /identity/v3/projects/961f8057321f482d9a5d9202c7bc3c4c => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:18.298616 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943fb14b-9099-48d8-bba2-676f3400ae8b None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:18.342002 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:18.406001 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943fb14b-9099-48d8-bba2-676f3400ae8b None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:18.444117 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b7f0ef-d983-43bf-883c-f0bb437dcdf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:18.444972 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b7f0ef-d983-43bf-883c-f0bb437dcdf2 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:18.445396 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b7f0ef-d983-43bf-883c-f0bb437dcdf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:18.445810 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b7f0ef-d983-43bf-883c-f0bb437dcdf2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:18.446698 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b7f0ef-d983-43bf-883c-f0bb437dcdf2 None admin] RBAC: Adding query filter params (role.id=f51dfe78039041e3a947ea27cfe97651, scope.project.id=961f8057321f482d9a5d9202c7bc3c4c, user.id=aa171c8644ee4d23babe25ebdaa1cf70) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:18.447148 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b7f0ef-d983-43bf-883c-f0bb437dcdf2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:18.448750 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b7f0ef-d983-43bf-883c-f0bb437dcdf2 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:18.456068 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 163/318] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:29:18 2022] GET /identity/v3/role_assignments?user.id=aa171c8644ee4d23babe25ebdaa1cf70&scope.project.id=961f8057321f482d9a5d9202c7bc3c4c&role.id=f51dfe78039041e3a947ea27cfe97651 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:18.798048 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0645d1-dc9d-4746-8cf4-87183ba63eed None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:18.798048 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0645d1-dc9d-4746-8cf4-87183ba63eed None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:18.798048 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0645d1-dc9d-4746-8cf4-87183ba63eed None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:18.798048 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 156/319] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:18.803359 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca81e80b-fa80-43fc-ad47-52d56ed15a03 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:18.803649 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca81e80b-fa80-43fc-ad47-52d56ed15a03 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:18.803922 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca81e80b-fa80-43fc-ad47-52d56ed15a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:18.822137 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-ca81e80b-fa80-43fc-ad47-52d56ed15a03 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:18.823336 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-ca81e80b-fa80-43fc-ad47-52d56ed15a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:18.891792 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 164/320] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:18.980682 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2513218a-7aa2-423b-8593-8a5bede78f3b None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:18.980978 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2513218a-7aa2-423b-8593-8a5bede78f3b None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:18.981419 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2513218a-7aa2-423b-8593-8a5bede78f3b None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:18.982764 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 157/321] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:18.988200 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09ff7c-6fb7-4f16-a429-f07ae7b25724 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:18.988631 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09ff7c-6fb7-4f16-a429-f07ae7b25724 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:18.988791 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09ff7c-6fb7-4f16-a429-f07ae7b25724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:19.008050 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-be09ff7c-6fb7-4f16-a429-f07ae7b25724 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:19.009259 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-be09ff7c-6fb7-4f16-a429-f07ae7b25724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:19.037781 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:19.081369 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 165/322] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:19.157896 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2513218a-7aa2-423b-8593-8a5bede78f3b None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:19.267590 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2513218a-7aa2-423b-8593-8a5bede78f3b None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:19.286015 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea7443d-2530-4e5e-a7f8-72be6612a535 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:19.286591 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea7443d-2530-4e5e-a7f8-72be6612a535 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:19.287075 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea7443d-2530-4e5e-a7f8-72be6612a535 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:19.288659 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 166/323] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:29:19.294568 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0663daf-f3c7-4fe3-b98d-98f225f706e6 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:19.295048 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0663daf-f3c7-4fe3-b98d-98f225f706e6 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:19.295355 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0663daf-f3c7-4fe3-b98d-98f225f706e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:19.306064 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f4ce2c-15bb-4315-a008-864520b3a03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:19.306700 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f4ce2c-15bb-4315-a008-864520b3a03b None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:19.306979 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f4ce2c-15bb-4315-a008-864520b3a03b None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:19.307782 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f4ce2c-15bb-4315-a008-864520b3a03b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:19.309729 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f4ce2c-15bb-4315-a008-864520b3a03b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:19.311489 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f4ce2c-15bb-4315-a008-864520b3a03b None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:19.315241 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 158/324] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:29:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:19.315379 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-b0663daf-f3c7-4fe3-b98d-98f225f706e6 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:19.315470 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-b0663daf-f3c7-4fe3-b98d-98f225f706e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:19.319294 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f4ce2c-15bb-4315-a008-864520b3a03b None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:19.345886 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:19.386717 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 167/325] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:19.391982 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0663daf-f3c7-4fe3-b98d-98f225f706e6 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:19.429891 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f4ce2c-15bb-4315-a008-864520b3a03b None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:19.464869 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a35e32-d434-4b1d-aff9-3d1c46e3c94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:19.465528 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a35e32-d434-4b1d-aff9-3d1c46e3c94a None admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:19.465949 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a35e32-d434-4b1d-aff9-3d1c46e3c94a None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:19.466226 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a35e32-d434-4b1d-aff9-3d1c46e3c94a None admin] PATH_INFO: `/v3/groups` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:19.466869 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a35e32-d434-4b1d-aff9-3d1c46e3c94a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:19.468342 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a35e32-d434-4b1d-aff9-3d1c46e3c94a None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:19.486268 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 159/326] 10.222.0.30 () {60 vars in 1134 bytes} [Tue Aug 30 14:29:19 2022] POST /identity/v3/groups => generated 226 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:29:19.488514 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:19.497571 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0663daf-f3c7-4fe3-b98d-98f225f706e6 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:19.536191 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da18241-e7a0-4271-99fb-a288cf8dfedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:19.536862 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da18241-e7a0-4271-99fb-a288cf8dfedb None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:19.537138 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da18241-e7a0-4271-99fb-a288cf8dfedb None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:19.537407 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da18241-e7a0-4271-99fb-a288cf8dfedb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:19.550048 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da18241-e7a0-4271-99fb-a288cf8dfedb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:19.555275 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-9da18241-e7a0-4271-99fb-a288cf8dfedb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:29:19.555275 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:19.555275 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:19.555275 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:19.555275 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:19.555275 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:19.555275 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:19.555275 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:19.555275 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:19.555275 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:19.555275 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:19.555275 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:19.555275 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:19.555275 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:19.555275 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:19.555275 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:19.556196 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:19.556196 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:19.556196 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:19.556196 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:19.556196 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:19.556196 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:19.556196 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:19.556196 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:19.556196 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:19.556196 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:19.556196 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:19.556196 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:19.556196 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:19.556196 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:19.556196 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:19.556196 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:19.557232 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:19.557232 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:19.557232 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:19.557232 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:19.557232 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:19.557232 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:19.557232 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:19.557232 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:19.557232 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:19.557232 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:19.557232 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:29:19.557232 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:19.557232 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 168/327] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:29:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:19.563290 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a35e32-d434-4b1d-aff9-3d1c46e3c94a None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:19.658866 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:19.662246 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da18241-e7a0-4271-99fb-a288cf8dfedb None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:19.678055 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a35e32-d434-4b1d-aff9-3d1c46e3c94a None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:19.715854 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e2a04c-3125-461b-9e52-8c10bcb30945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:19.716597 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e2a04c-3125-461b-9e52-8c10bcb30945 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:19.716893 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e2a04c-3125-461b-9e52-8c10bcb30945 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:19.717150 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e2a04c-3125-461b-9e52-8c10bcb30945 None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:19.717822 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e2a04c-3125-461b-9e52-8c10bcb30945 None admin] RBAC: Adding query filter params (name=member) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:19.718121 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e2a04c-3125-461b-9e52-8c10bcb30945 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:19.720410 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e2a04c-3125-461b-9e52-8c10bcb30945 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:19.728216 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 160/328] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 30 14:29:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:19.734365 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e2a04c-3125-461b-9e52-8c10bcb30945 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:19.794491 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da18241-e7a0-4271-99fb-a288cf8dfedb None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:19.842619 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc81f47c-8e20-4f62-9471-89c811108e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:19.843674 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc81f47c-8e20-4f62-9471-89c811108e94 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:19.844137 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc81f47c-8e20-4f62-9471-89c811108e94 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:19.844637 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc81f47c-8e20-4f62-9471-89c811108e94 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:19.857653 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc81f47c-8e20-4f62-9471-89c811108e94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:19.863300 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-fc81f47c-8e20-4f62-9471-89c811108e94 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:29:19.863300 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:19.863300 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:19.863300 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:19.863300 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:19.863300 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:19.863300 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:19.863300 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:19.863300 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:19.863300 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:19.863300 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:19.863300 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:19.863300 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:19.863300 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:19.863300 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:19.863300 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:19.864198 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:19.864198 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:19.864198 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:19.864198 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:19.864198 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:19.864198 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:19.864198 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:19.864198 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:19.864198 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:19.864198 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:19.864198 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:19.864198 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:19.864198 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:19.864198 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:19.864198 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:19.864198 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:19.864997 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:19.864997 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:19.864997 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:19.864997 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:19.864997 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:19.864997 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:19.864997 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:19.864997 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:19.864997 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:19.864997 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:19.864997 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:29:19.864997 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:19.866721 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 169/329] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:29:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:19.872241 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc81f47c-8e20-4f62-9471-89c811108e94 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:19.874530 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e2a04c-3125-461b-9e52-8c10bcb30945 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:19.892221 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:19.921357 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e860ac-7d2e-4467-a89b-0cd423e68439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:19.921983 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e860ac-7d2e-4467-a89b-0cd423e68439 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:19.922236 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e860ac-7d2e-4467-a89b-0cd423e68439 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:19.922509 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e860ac-7d2e-4467-a89b-0cd423e68439 None admin] PATH_INFO: `/v3/users/e4b454e73b9d437e969b865e5f7f23a1` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:19.923732 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e860ac-7d2e-4467-a89b-0cd423e68439 None admin] RBAC: Authorizing `identity:get_user(user_id=e4b454e73b9d437e969b865e5f7f23a1)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:19.925196 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e860ac-7d2e-4467-a89b-0cd423e68439 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:19.926793 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 161/330] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:29:19 2022] GET /identity/v3/users/e4b454e73b9d437e969b865e5f7f23a1 => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:19.932602 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e860ac-7d2e-4467-a89b-0cd423e68439 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:19.955576 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:20.024573 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc81f47c-8e20-4f62-9471-89c811108e94 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:20.065833 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e860ac-7d2e-4467-a89b-0cd423e68439 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:20.078878 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a387ac-59ff-442e-bc41-df2da300cfbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:20.079613 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a387ac-59ff-442e-bc41-df2da300cfbe None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:20.079891 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a387ac-59ff-442e-bc41-df2da300cfbe None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:20.080152 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a387ac-59ff-442e-bc41-df2da300cfbe None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:20.082833 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a387ac-59ff-442e-bc41-df2da300cfbe None admin] RBAC: Adding query filter params (name=reader) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:20.082833 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a387ac-59ff-442e-bc41-df2da300cfbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:20.082833 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a387ac-59ff-442e-bc41-df2da300cfbe None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:20.092620 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 170/331] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 30 14:29:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:20.097672 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a387ac-59ff-442e-bc41-df2da300cfbe None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:20.110537 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67df439-3d6c-4cb2-9dd9-aaeaeadc439f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:20.111521 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67df439-3d6c-4cb2-9dd9-aaeaeadc439f None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:20.111939 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67df439-3d6c-4cb2-9dd9-aaeaeadc439f None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:20.112370 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67df439-3d6c-4cb2-9dd9-aaeaeadc439f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:20.113335 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67df439-3d6c-4cb2-9dd9-aaeaeadc439f None admin] RBAC: Adding query filter params (role.id=6a985d5659a3416cba33ca9d30b39f85, user.id=e4b454e73b9d437e969b865e5f7f23a1, scope.system=all) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:20.113772 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67df439-3d6c-4cb2-9dd9-aaeaeadc439f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:20.115402 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67df439-3d6c-4cb2-9dd9-aaeaeadc439f None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:20.129020 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 162/332] 10.222.0.30 () {56 vars in 1308 bytes} [Tue Aug 30 14:29:19 2022] GET /identity/v3/role_assignments?user.id=e4b454e73b9d437e969b865e5f7f23a1&scope.system=all&role.id=6a985d5659a3416cba33ca9d30b39f85 => generated 510 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:20.212287 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:20.241849 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a387ac-59ff-442e-bc41-df2da300cfbe None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:20.283320 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2ce834-1216-4ed8-858e-74c3472c9169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:20.284024 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ce834-1216-4ed8-858e-74c3472c9169 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:20.284306 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ce834-1216-4ed8-858e-74c3472c9169 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:20.284577 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ce834-1216-4ed8-858e-74c3472c9169 None admin] PATH_INFO: `/v3/users/999b71fa2e67451ba9d3140167051c4f` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:20.285836 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2ce834-1216-4ed8-858e-74c3472c9169 None admin] RBAC: Authorizing `identity:get_user(user_id=999b71fa2e67451ba9d3140167051c4f)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:20.287332 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2ce834-1216-4ed8-858e-74c3472c9169 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:20.289320 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 171/333] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:29:20 2022] GET /identity/v3/users/999b71fa2e67451ba9d3140167051c4f => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:20.294881 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67df439-3d6c-4cb2-9dd9-aaeaeadc439f None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:20.386385 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:20.420743 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67df439-3d6c-4cb2-9dd9-aaeaeadc439f None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:20.462202 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b0df5d-2d3b-45ab-9edc-866ac41eb4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:20.463193 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b0df5d-2d3b-45ab-9edc-866ac41eb4de None admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:20.463757 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b0df5d-2d3b-45ab-9edc-866ac41eb4de None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:20.464232 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b0df5d-2d3b-45ab-9edc-866ac41eb4de None admin] PATH_INFO: `/v3/system/users/999b71fa2e67451ba9d3140167051c4f/roles/f51dfe78039041e3a947ea27cfe97651` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:20.466232 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b0df5d-2d3b-45ab-9edc-866ac41eb4de None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=999b71fa2e67451ba9d3140167051c4f, role_id=f51dfe78039041e3a947ea27cfe97651)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:20.467889 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b0df5d-2d3b-45ab-9edc-866ac41eb4de None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:20.478583 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 163/334] 10.222.0.30 () {58 vars in 1334 bytes} [Tue Aug 30 14:29:20 2022] PUT /identity/v3/system/users/999b71fa2e67451ba9d3140167051c4f/roles/f51dfe78039041e3a947ea27cfe97651 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:29:22.094618 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a3e77b-1ca7-4129-95a9-70a80139e79b None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:22.095164 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a3e77b-1ca7-4129-95a9-70a80139e79b None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:22.095164 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a3e77b-1ca7-4129-95a9-70a80139e79b None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:22.096856 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 172/335] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:22.102165 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db1ca01-64db-45d6-ace9-3069bee978ca None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:22.103115 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db1ca01-64db-45d6-ace9-3069bee978ca None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:22.103623 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db1ca01-64db-45d6-ace9-3069bee978ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:22.127604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-2db1ca01-64db-45d6-ace9-3069bee978ca None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:22.129166 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-2db1ca01-64db-45d6-ace9-3069bee978ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:22.223237 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 164/336] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:22.356982 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87174a-84a6-45a7-8264-726386e52eb7 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:22.357588 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87174a-84a6-45a7-8264-726386e52eb7 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:22.358078 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87174a-84a6-45a7-8264-726386e52eb7 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:22.359893 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 173/337] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:22.365543 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe5c26-f9b6-45b0-91d0-3ebbfd28e713 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:22.366057 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe5c26-f9b6-45b0-91d0-3ebbfd28e713 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:22.366538 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe5c26-f9b6-45b0-91d0-3ebbfd28e713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:22.392195 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-4bbe5c26-f9b6-45b0-91d0-3ebbfd28e713 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:22.394002 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-4bbe5c26-f9b6-45b0-91d0-3ebbfd28e713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:22.460141 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:22.470484 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 165/338] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:22.596223 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb87174a-84a6-45a7-8264-726386e52eb7 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:22.651387 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cdfc20-d282-4ce0-8efa-ec9dc6513412 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:22.651996 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cdfc20-d282-4ce0-8efa-ec9dc6513412 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:22.652584 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cdfc20-d282-4ce0-8efa-ec9dc6513412 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:22.655310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 166/339] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:29:22.659950 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc3a02-a1b4-442e-9b1f-487c2c23acb6 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:22.660477 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc3a02-a1b4-442e-9b1f-487c2c23acb6 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:22.660982 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc3a02-a1b4-442e-9b1f-487c2c23acb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:22.680529 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-40bc3a02-a1b4-442e-9b1f-487c2c23acb6 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:22.682130 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-40bc3a02-a1b4-442e-9b1f-487c2c23acb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:22.750627 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb87174a-84a6-45a7-8264-726386e52eb7 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:22.810430 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 167/340] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:29:22.825315 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3febbc-5d75-416d-8803-e74da3ca27d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:22.826323 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3febbc-5d75-416d-8803-e74da3ca27d6 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:22.826831 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3febbc-5d75-416d-8803-e74da3ca27d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:22.827360 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3febbc-5d75-416d-8803-e74da3ca27d6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:22.848148 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3febbc-5d75-416d-8803-e74da3ca27d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:22.853861 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-6c3febbc-5d75-416d-8803-e74da3ca27d6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:29:22.853861 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:22.853861 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:22.853861 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:22.853861 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:22.853861 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:22.853861 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:22.853861 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:22.853861 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:22.853861 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:22.853861 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:22.853861 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:22.853861 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:22.853861 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:22.853861 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:22.853861 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:22.854746 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:22.854746 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:22.854746 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:22.854746 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:22.854746 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:22.854746 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:22.854746 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:22.854746 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:22.854746 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:22.854746 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:22.854746 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:22.854746 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:22.854746 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:22.854746 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:22.854746 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:22.854746 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:22.855636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:22.855636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:22.855636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:22.855636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:22.855636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:22.855636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:22.855636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:22.855636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:22.855636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:22.855636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:22.855636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:29:22.855636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:22.862636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 174/341] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:29:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:22.863953 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bc3a02-a1b4-442e-9b1f-487c2c23acb6 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:22.908643 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3febbc-5d75-416d-8803-e74da3ca27d6 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:22.915874 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:23.055040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bc3a02-a1b4-442e-9b1f-487c2c23acb6 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:23.102031 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3febbc-5d75-416d-8803-e74da3ca27d6 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:23.106071 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:23.125357 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f3da69-65be-4799-83e1-8ea99d2a625b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:23.125986 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3da69-65be-4799-83e1-8ea99d2a625b None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:23.126265 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3da69-65be-4799-83e1-8ea99d2a625b None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:23.126536 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3da69-65be-4799-83e1-8ea99d2a625b None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:23.127224 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f3da69-65be-4799-83e1-8ea99d2a625b None admin] RBAC: Adding query filter params (name=member) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:23.127528 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f3da69-65be-4799-83e1-8ea99d2a625b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:23.128915 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f3da69-65be-4799-83e1-8ea99d2a625b None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:23.149641 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 168/342] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 30 14:29:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:23.161889 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f3da69-65be-4799-83e1-8ea99d2a625b None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:23.172551 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e89ea-2caa-465b-be63-e93406570624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:23.173212 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e89ea-2caa-465b-be63-e93406570624 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:23.173496 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e89ea-2caa-465b-be63-e93406570624 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:23.173776 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e89ea-2caa-465b-be63-e93406570624 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:23.197394 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484e89ea-2caa-465b-be63-e93406570624 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:23.203003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-484e89ea-2caa-465b-be63-e93406570624 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:29:23.203003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:23.203003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:23.203003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:23.203003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:23.203003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:23.203003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:23.203003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:23.203003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:23.203003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:23.203003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:23.203003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:23.203003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:23.203003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:23.203003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:23.203003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:23.204127 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:23.204127 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:23.204127 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:23.204127 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:23.204127 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:23.204127 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:23.204127 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:23.204127 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:23.204127 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:23.204127 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:23.204127 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:23.204127 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:23.204127 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:23.204127 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:23.204127 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:23.204127 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:23.205116 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:23.205116 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:23.205116 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:23.205116 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:23.205116 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:23.205116 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:23.205116 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:23.205116 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:23.205116 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:23.205116 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:23.205116 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:29:23.205116 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:23.207456 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 175/343] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 30 14:29:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:29:23.215250 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e89ea-2caa-465b-be63-e93406570624 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:23.343734 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:23.407489 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f3da69-65be-4799-83e1-8ea99d2a625b None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:23.409054 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e89ea-2caa-465b-be63-e93406570624 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:23.453414 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa1e0c9-4eaa-417d-9765-59b897cd5008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:23.453959 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa1e0c9-4eaa-417d-9765-59b897cd5008 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:23.454239 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa1e0c9-4eaa-417d-9765-59b897cd5008 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:23.454509 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa1e0c9-4eaa-417d-9765-59b897cd5008 None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:23.455181 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa1e0c9-4eaa-417d-9765-59b897cd5008 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:23.455511 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa1e0c9-4eaa-417d-9765-59b897cd5008 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:23.456906 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa1e0c9-4eaa-417d-9765-59b897cd5008 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:23.466559 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 176/344] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:29:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:29:23.467439 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa1e0c9-4eaa-417d-9765-59b897cd5008 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:23.482309 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88967624-d6f3-4530-a77b-431ddfb56c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:23.482938 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88967624-d6f3-4530-a77b-431ddfb56c83 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:23.483240 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88967624-d6f3-4530-a77b-431ddfb56c83 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:23.483513 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88967624-d6f3-4530-a77b-431ddfb56c83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:23.581078 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa1e0c9-4eaa-417d-9765-59b897cd5008 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:23.587579 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:23.613232 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88967624-d6f3-4530-a77b-431ddfb56c83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:23.614713 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88967624-d6f3-4530-a77b-431ddfb56c83 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:23.642058 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2477ae-1596-495e-8659-170388b13817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:23.642721 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2477ae-1596-495e-8659-170388b13817 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:23.643044 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2477ae-1596-495e-8659-170388b13817 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:23.643359 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2477ae-1596-495e-8659-170388b13817 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:23.656374 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2477ae-1596-495e-8659-170388b13817 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:23.662838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-fb2477ae-1596-495e-8659-170388b13817 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:29:23.662838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:23.662838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:23.662838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:23.662838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:23.662838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:23.662838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:23.662838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:23.662838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:23.662838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:23.662838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:23.662838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:23.662838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:23.662838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:23.662838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:23.662838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:23.663823 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:23.663823 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:23.663823 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:23.663823 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:23.663823 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:23.663823 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:23.663823 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:23.663823 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:23.663823 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:23.663823 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:23.663823 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:23.663823 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:23.663823 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:23.663823 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:23.663823 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:23.663823 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:23.664630 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:23.664630 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:23.664630 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:23.664630 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:23.664630 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:23.664630 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:23.664630 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:23.664630 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:23.664630 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:23.664630 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:23.664630 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:29:23.664630 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:23.664630 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 177/345] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:29:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:23.665305 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2477ae-1596-495e-8659-170388b13817 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:23.671261 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:23.779333 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2477ae-1596-495e-8659-170388b13817 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:23.779501 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 169/346] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:29:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 618 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 14:29:23.780373 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88967624-d6f3-4530-a77b-431ddfb56c83 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:23.815393 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea1c25-8e5b-48b6-b368-44678c38608a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:23.816133 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea1c25-8e5b-48b6-b368-44678c38608a None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:23.816417 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea1c25-8e5b-48b6-b368-44678c38608a None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:23.816688 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea1c25-8e5b-48b6-b368-44678c38608a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:23.888231 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:23.893922 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:23.906365 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ea1c25-8e5b-48b6-b368-44678c38608a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:23.908168 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ea1c25-8e5b-48b6-b368-44678c38608a None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:23.913038 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88967624-d6f3-4530-a77b-431ddfb56c83 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:23.958957 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9163ff74-63a9-4f24-81ab-29f876b60dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:23.959939 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163ff74-63a9-4f24-81ab-29f876b60dd0 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:23.960421 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163ff74-63a9-4f24-81ab-29f876b60dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:23.960989 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163ff74-63a9-4f24-81ab-29f876b60dd0 None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:23.961870 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9163ff74-63a9-4f24-81ab-29f876b60dd0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:23.962250 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9163ff74-63a9-4f24-81ab-29f876b60dd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:23.963732 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9163ff74-63a9-4f24-81ab-29f876b60dd0 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:23.972716 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 170/347] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 30 14:29:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:23.974796 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054ef20-d4da-465b-8226-6d35a3388173 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:23.975090 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054ef20-d4da-465b-8226-6d35a3388173 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:23.975393 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054ef20-d4da-465b-8226-6d35a3388173 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:23.976924 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 171/348] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:23.978272 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c326183c-814d-4674-9276-a05a10b3f752 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:23.978586 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c326183c-814d-4674-9276-a05a10b3f752 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:23.978823 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c326183c-814d-4674-9276-a05a10b3f752 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:23.980129 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 172/349] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:23.984449 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c326183c-814d-4674-9276-a05a10b3f752 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:24.031559 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 178/350] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:29:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:29:24.032341 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea1c25-8e5b-48b6-b368-44678c38608a None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:24.109795 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c326183c-814d-4674-9276-a05a10b3f752 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:24.137927 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:24.149791 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a63ae3-e083-440b-a16c-866dc79d7327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:24.151524 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a63ae3-e083-440b-a16c-866dc79d7327 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:24.151524 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a63ae3-e083-440b-a16c-866dc79d7327 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:24.151524 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a63ae3-e083-440b-a16c-866dc79d7327 None admin] PATH_INFO: `/v3/projects/1c2139beef6b44b9a0574047298c1af9` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:24.153489 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a63ae3-e083-440b-a16c-866dc79d7327 None admin] RBAC: Authorizing `identity:get_project(project_id=1c2139beef6b44b9a0574047298c1af9)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:24.155010 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a63ae3-e083-440b-a16c-866dc79d7327 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:24.156262 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 173/351] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:29:23 2022] GET /identity/v3/projects/1c2139beef6b44b9a0574047298c1af9 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:24.157234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b13cc-632c-4062-b036-5e9795cbeffb None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:24.157533 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b13cc-632c-4062-b036-5e9795cbeffb None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:24.157633 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea1c25-8e5b-48b6-b368-44678c38608a None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:24.157771 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b13cc-632c-4062-b036-5e9795cbeffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:24.176572 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-832b13cc-632c-4062-b036-5e9795cbeffb None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:24.179269 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-832b13cc-632c-4062-b036-5e9795cbeffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:24.193842 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:24.204206 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89487056-f445-478c-9bc7-bdd361e8cc50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:24.205089 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89487056-f445-478c-9bc7-bdd361e8cc50 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:24.205375 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89487056-f445-478c-9bc7-bdd361e8cc50 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:24.205652 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89487056-f445-478c-9bc7-bdd361e8cc50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:24.264036 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 174/352] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:24.265103 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cef49e-fe56-4299-9a84-df57b92b245e None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:24.265388 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cef49e-fe56-4299-9a84-df57b92b245e None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:24.265668 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cef49e-fe56-4299-9a84-df57b92b245e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:24.285610 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-95cef49e-fe56-4299-9a84-df57b92b245e None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:24.286742 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-95cef49e-fe56-4299-9a84-df57b92b245e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:24.297207 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89487056-f445-478c-9bc7-bdd361e8cc50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:24.298686 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89487056-f445-478c-9bc7-bdd361e8cc50 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:24.377514 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 175/353] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:24.379255 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7675bcf-67e8-4bcb-b7b7-141d42472da2 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:24.379255 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7675bcf-67e8-4bcb-b7b7-141d42472da2 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:24.379449 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7675bcf-67e8-4bcb-b7b7-141d42472da2 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:24.380608 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 176/354] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:24.381310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7675bcf-67e8-4bcb-b7b7-141d42472da2 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:24.436507 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:24.455597 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 179/355] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:29:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:29:24.456255 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89487056-f445-478c-9bc7-bdd361e8cc50 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:24.487119 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:24.523077 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7675bcf-67e8-4bcb-b7b7-141d42472da2 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:24.565559 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ed76dd-25d9-4ac0-b20b-22dc758ee71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:24.566202 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed76dd-25d9-4ac0-b20b-22dc758ee71f None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:24.566478 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed76dd-25d9-4ac0-b20b-22dc758ee71f None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:24.566747 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed76dd-25d9-4ac0-b20b-22dc758ee71f None admin] PATH_INFO: `/v3/projects/1c2139beef6b44b9a0574047298c1af9` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:24.568836 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ed76dd-25d9-4ac0-b20b-22dc758ee71f None admin] RBAC: Authorizing `identity:get_project(project_id=1c2139beef6b44b9a0574047298c1af9)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:24.572211 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ed76dd-25d9-4ac0-b20b-22dc758ee71f None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:24.573190 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 177/356] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:29:24 2022] GET /identity/v3/projects/1c2139beef6b44b9a0574047298c1af9 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:24.578208 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b743f65a-94d9-4d96-ab10-0d2d091075d7 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:24.578208 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b743f65a-94d9-4d96-ab10-0d2d091075d7 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:24.578208 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b743f65a-94d9-4d96-ab10-0d2d091075d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:24.578466 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89487056-f445-478c-9bc7-bdd361e8cc50 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:24.599894 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-b743f65a-94d9-4d96-ab10-0d2d091075d7 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:24.601541 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-b743f65a-94d9-4d96-ab10-0d2d091075d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:24.624110 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef48695-4d62-44e0-9368-684baf602add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:24.624801 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef48695-4d62-44e0-9368-684baf602add None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:24.625076 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef48695-4d62-44e0-9368-684baf602add None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:24.625345 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef48695-4d62-44e0-9368-684baf602add None admin] PATH_INFO: `/v3/groups/88ad07bcdacd4ef79be074cf74d7f8b2` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:24.632208 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef48695-4d62-44e0-9368-684baf602add None admin] RBAC: Authorizing `identity:get_group(group_id=88ad07bcdacd4ef79be074cf74d7f8b2)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:24.633784 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef48695-4d62-44e0-9368-684baf602add None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:24.635880 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 180/357] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Aug 30 14:29:24 2022] GET /identity/v3/groups/88ad07bcdacd4ef79be074cf74d7f8b2 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:24.636549 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef48695-4d62-44e0-9368-684baf602add None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:24.690581 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 178/358] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:24.691431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b743f65a-94d9-4d96-ab10-0d2d091075d7 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:24.713961 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:24.773133 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef48695-4d62-44e0-9368-684baf602add None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:24.784345 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:24.814291 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb8cdb3-2d12-498e-99ec-cd691fef9973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:24.815114 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b743f65a-94d9-4d96-ab10-0d2d091075d7 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:24.815428 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb8cdb3-2d12-498e-99ec-cd691fef9973 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:24.815918 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb8cdb3-2d12-498e-99ec-cd691fef9973 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:24.816348 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb8cdb3-2d12-498e-99ec-cd691fef9973 None admin] PATH_INFO: `/v3/projects/961f8057321f482d9a5d9202c7bc3c4c` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:24.818132 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb8cdb3-2d12-498e-99ec-cd691fef9973 None admin] RBAC: Authorizing `identity:get_project(project_id=961f8057321f482d9a5d9202c7bc3c4c)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:24.819617 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb8cdb3-2d12-498e-99ec-cd691fef9973 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:24.821889 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 181/359] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:29:24 2022] GET /identity/v3/projects/961f8057321f482d9a5d9202c7bc3c4c => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:24.822164 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb8cdb3-2d12-498e-99ec-cd691fef9973 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:24.861560 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2408504-f0dc-4a87-bbd9-8c4d9d155e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:24.862210 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2408504-f0dc-4a87-bbd9-8c4d9d155e65 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:24.862490 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2408504-f0dc-4a87-bbd9-8c4d9d155e65 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:24.862760 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2408504-f0dc-4a87-bbd9-8c4d9d155e65 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:24.876265 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2408504-f0dc-4a87-bbd9-8c4d9d155e65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:24.883251 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-e2408504-f0dc-4a87-bbd9-8c4d9d155e65 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:29:24.883251 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:24.883251 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:24.883251 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:24.883251 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:24.883251 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:24.883251 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:24.883251 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:24.883251 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:24.883251 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:24.883251 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:24.883251 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:24.883251 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:24.883251 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:24.883251 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:24.883251 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:24.884143 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:24.884143 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:24.884143 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:24.884143 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:24.884143 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:24.884143 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:24.884143 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:24.884143 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:24.884143 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:24.884143 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:24.884143 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:24.884143 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:24.884143 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:24.884143 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:24.884143 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:24.884143 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:24.884922 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:24.884922 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:24.884922 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:24.884922 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:24.884922 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:24.884922 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:24.884922 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:24.884922 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:24.884922 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:24.884922 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:24.884922 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 14:29:24.884922 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:24.884922 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 179/360] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:29:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:24.885615 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2408504-f0dc-4a87-bbd9-8c4d9d155e65 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:24.948230 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb8cdb3-2d12-498e-99ec-cd691fef9973 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:24.973452 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:24.996983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ed8b0a-85f3-4807-aefc-2070e607395a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:24.997406 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed8b0a-85f3-4807-aefc-2070e607395a None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:24.997687 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed8b0a-85f3-4807-aefc-2070e607395a None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:24.997960 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed8b0a-85f3-4807-aefc-2070e607395a None admin] PATH_INFO: `/v3/groups/88ad07bcdacd4ef79be074cf74d7f8b2` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:24.999254 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ed8b0a-85f3-4807-aefc-2070e607395a None admin] RBAC: Authorizing `identity:get_group(group_id=88ad07bcdacd4ef79be074cf74d7f8b2)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:25.000571 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ed8b0a-85f3-4807-aefc-2070e607395a None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:25.002167 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 182/361] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Aug 30 14:29:24 2022] GET /identity/v3/groups/88ad07bcdacd4ef79be074cf74d7f8b2 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:25.003796 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ed8b0a-85f3-4807-aefc-2070e607395a None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:25.017070 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2408504-f0dc-4a87-bbd9-8c4d9d155e65 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:25.061275 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8c9514-1485-48ec-9da7-09e37218ff70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:25.061891 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c9514-1485-48ec-9da7-09e37218ff70 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:25.062168 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c9514-1485-48ec-9da7-09e37218ff70 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:25.062436 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c9514-1485-48ec-9da7-09e37218ff70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:25.063166 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8c9514-1485-48ec-9da7-09e37218ff70 None admin] RBAC: Adding query filter params (group.id=88ad07bcdacd4ef79be074cf74d7f8b2, role.id=6a985d5659a3416cba33ca9d30b39f85, scope.project.id=1c2139beef6b44b9a0574047298c1af9) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:25.063946 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8c9514-1485-48ec-9da7-09e37218ff70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:25.064946 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8c9514-1485-48ec-9da7-09e37218ff70 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:25.068013 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:25.071041 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 180/362] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Aug 30 14:29:24 2022] GET /identity/v3/role_assignments?group.id=88ad07bcdacd4ef79be074cf74d7f8b2&scope.project.id=1c2139beef6b44b9a0574047298c1af9&role.id=6a985d5659a3416cba33ca9d30b39f85 => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:25.071792 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8c9514-1485-48ec-9da7-09e37218ff70 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:25.140159 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ed8b0a-85f3-4807-aefc-2070e607395a None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:25.187410 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d372932-8d9e-4a57-93ef-91b3845eb000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:25.188134 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d372932-8d9e-4a57-93ef-91b3845eb000 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:25.188418 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d372932-8d9e-4a57-93ef-91b3845eb000 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:25.188689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d372932-8d9e-4a57-93ef-91b3845eb000 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:25.202352 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d372932-8d9e-4a57-93ef-91b3845eb000 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:25.207856 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-3d372932-8d9e-4a57-93ef-91b3845eb000 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:29:25.207856 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:25.207856 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:25.207856 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:25.207856 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:25.207856 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:25.207856 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:25.207856 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:25.207856 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:25.207856 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:25.207856 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:25.207856 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:25.207856 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:25.207856 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:25.207856 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:25.207856 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:25.208809 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:25.208809 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:25.208809 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:25.208809 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:25.208809 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:25.208809 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:25.208809 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:25.208809 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:25.208809 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:25.208809 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:25.208809 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:25.208809 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:25.208809 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:25.208809 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:25.208809 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:25.208809 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:25.209700 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:25.209700 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:25.209700 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:25.209700 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:25.209700 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:25.209700 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:25.209700 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:25.209700 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:25.209700 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:25.209700 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:25.209700 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:29:25.209700 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:25.209700 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 183/363] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 30 14:29:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:25.210371 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d372932-8d9e-4a57-93ef-91b3845eb000 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:25.231097 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8c9514-1485-48ec-9da7-09e37218ff70 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:25.271322 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:25.281101 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef853edb-90c7-4e7a-931a-9fc7ce9c2c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:25.281760 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef853edb-90c7-4e7a-931a-9fc7ce9c2c8b None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:25.282041 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef853edb-90c7-4e7a-931a-9fc7ce9c2c8b None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:25.282314 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef853edb-90c7-4e7a-931a-9fc7ce9c2c8b None admin] PATH_INFO: `/v3/groups/88ad07bcdacd4ef79be074cf74d7f8b2` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:25.286199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef853edb-90c7-4e7a-931a-9fc7ce9c2c8b None admin] RBAC: Authorizing `identity:get_group(group_id=88ad07bcdacd4ef79be074cf74d7f8b2)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:25.287708 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef853edb-90c7-4e7a-931a-9fc7ce9c2c8b None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:25.289323 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 181/364] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Aug 30 14:29:25 2022] GET /identity/v3/groups/88ad07bcdacd4ef79be074cf74d7f8b2 => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:25.290077 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef853edb-90c7-4e7a-931a-9fc7ce9c2c8b None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:25.348918 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d372932-8d9e-4a57-93ef-91b3845eb000 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:25.384598 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:25.391008 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1189cf6-d2c7-4e22-ae81-d8337f3e2cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:25.391646 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1189cf6-d2c7-4e22-ae81-d8337f3e2cb1 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:25.391920 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1189cf6-d2c7-4e22-ae81-d8337f3e2cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:25.392201 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1189cf6-d2c7-4e22-ae81-d8337f3e2cb1 None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:25.392878 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1189cf6-d2c7-4e22-ae81-d8337f3e2cb1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:25.393188 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1189cf6-d2c7-4e22-ae81-d8337f3e2cb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:25.394580 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1189cf6-d2c7-4e22-ae81-d8337f3e2cb1 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:25.404348 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 184/365] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 30 14:29:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:25.405106 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1189cf6-d2c7-4e22-ae81-d8337f3e2cb1 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:25.439125 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef853edb-90c7-4e7a-931a-9fc7ce9c2c8b None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:25.491104 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71f7b14-3ff9-42c8-ad86-4ab962361a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:25.491788 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f7b14-3ff9-42c8-ad86-4ab962361a4a None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:25.492065 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f7b14-3ff9-42c8-ad86-4ab962361a4a None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:25.492336 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f7b14-3ff9-42c8-ad86-4ab962361a4a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:25.506444 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71f7b14-3ff9-42c8-ad86-4ab962361a4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:25.515835 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-c71f7b14-3ff9-42c8-ad86-4ab962361a4a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:29:25.515835 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:25.515835 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:25.515835 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:25.515835 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:25.515835 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:25.515835 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:25.515835 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:25.515835 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:25.515835 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:25.515835 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:25.515835 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:25.515835 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:25.515835 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:25.515835 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:25.515835 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:25.516754 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:25.516754 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:25.516754 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:25.516754 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:25.516754 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:25.516754 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:25.516754 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:25.516754 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:25.516754 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:25.516754 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:25.516754 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:25.516754 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:25.516754 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:25.516754 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:25.516754 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:25.516754 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:25.517559 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:25.517559 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:25.517559 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:25.517559 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:25.517559 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:25.517559 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:25.517559 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:25.517559 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:25.517559 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:25.517559 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:25.517559 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:29:25.517559 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:25.517559 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 182/366] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 30 14:29:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:25.518239 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71f7b14-3ff9-42c8-ad86-4ab962361a4a None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:25.547506 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1189cf6-d2c7-4e22-ae81-d8337f3e2cb1 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:25.560606 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:25.595061 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b8d866-f8b3-4b39-9402-b4eb3f3ee070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:25.596813 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8d866-f8b3-4b39-9402-b4eb3f3ee070 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:25.596813 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8d866-f8b3-4b39-9402-b4eb3f3ee070 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:25.596813 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8d866-f8b3-4b39-9402-b4eb3f3ee070 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:25.597229 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b8d866-f8b3-4b39-9402-b4eb3f3ee070 None admin] RBAC: Adding query filter params (group.id=88ad07bcdacd4ef79be074cf74d7f8b2, role.id=1c4886a7a7fd402d8221a0158ddcb167, scope.project.id=1c2139beef6b44b9a0574047298c1af9) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:25.597608 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b8d866-f8b3-4b39-9402-b4eb3f3ee070 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:25.599416 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b8d866-f8b3-4b39-9402-b4eb3f3ee070 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:25.605252 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 185/367] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Aug 30 14:29:25 2022] GET /identity/v3/role_assignments?group.id=88ad07bcdacd4ef79be074cf74d7f8b2&scope.project.id=1c2139beef6b44b9a0574047298c1af9&role.id=1c4886a7a7fd402d8221a0158ddcb167 => generated 260 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:25.606134 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b8d866-f8b3-4b39-9402-b4eb3f3ee070 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:25.641759 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71f7b14-3ff9-42c8-ad86-4ab962361a4a None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:25.673234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:25.700510 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016f036d-fa71-4f7f-8c79-95fe815e136c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:25.701588 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f036d-fa71-4f7f-8c79-95fe815e136c None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:25.702370 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f036d-fa71-4f7f-8c79-95fe815e136c None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:25.703551 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f036d-fa71-4f7f-8c79-95fe815e136c None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:25.704851 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016f036d-fa71-4f7f-8c79-95fe815e136c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:25.705389 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016f036d-fa71-4f7f-8c79-95fe815e136c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:25.707788 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016f036d-fa71-4f7f-8c79-95fe815e136c None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:25.725223 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 183/368] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:29:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:25.725223 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016f036d-fa71-4f7f-8c79-95fe815e136c None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:25.735121 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b8d866-f8b3-4b39-9402-b4eb3f3ee070 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:25.790752 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0819bc5-da88-47c0-a75d-2bd8166ad8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:25.791559 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0819bc5-da88-47c0-a75d-2bd8166ad8a2 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:25.791972 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0819bc5-da88-47c0-a75d-2bd8166ad8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:25.792347 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0819bc5-da88-47c0-a75d-2bd8166ad8a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:25.793383 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0819bc5-da88-47c0-a75d-2bd8166ad8a2 None admin] RBAC: Adding query filter params (group.id=88ad07bcdacd4ef79be074cf74d7f8b2, role.id=6a985d5659a3416cba33ca9d30b39f85, scope.project.id=961f8057321f482d9a5d9202c7bc3c4c) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:25.793745 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0819bc5-da88-47c0-a75d-2bd8166ad8a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:25.795384 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0819bc5-da88-47c0-a75d-2bd8166ad8a2 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:25.807289 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 186/369] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Aug 30 14:29:25 2022] GET /identity/v3/role_assignments?group.id=88ad07bcdacd4ef79be074cf74d7f8b2&scope.project.id=961f8057321f482d9a5d9202c7bc3c4c&role.id=6a985d5659a3416cba33ca9d30b39f85 => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:25.807289 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0819bc5-da88-47c0-a75d-2bd8166ad8a2 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:25.863732 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:25.941283 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016f036d-fa71-4f7f-8c79-95fe815e136c None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:25.946278 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0819bc5-da88-47c0-a75d-2bd8166ad8a2 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:25.999020 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3f390f-1da2-4439-91a1-17d028841ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:25.999683 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f390f-1da2-4439-91a1-17d028841ec3 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:25.999962 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f390f-1da2-4439-91a1-17d028841ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:26.000235 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f390f-1da2-4439-91a1-17d028841ec3 None admin] PATH_INFO: `/v3/users/999b71fa2e67451ba9d3140167051c4f` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:26.001700 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3f390f-1da2-4439-91a1-17d028841ec3 None admin] RBAC: Authorizing `identity:get_user(user_id=999b71fa2e67451ba9d3140167051c4f)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:26.003329 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3f390f-1da2-4439-91a1-17d028841ec3 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:26.005184 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 184/370] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:29:25 2022] GET /identity/v3/users/999b71fa2e67451ba9d3140167051c4f => generated 293 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:26.005660 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3f390f-1da2-4439-91a1-17d028841ec3 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:26.008902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c184b-90ab-47ae-8294-6f084404358f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:26.009518 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c184b-90ab-47ae-8294-6f084404358f None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:26.009793 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c184b-90ab-47ae-8294-6f084404358f None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:26.010063 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c184b-90ab-47ae-8294-6f084404358f None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:26.010732 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126c184b-90ab-47ae-8294-6f084404358f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:26.011029 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126c184b-90ab-47ae-8294-6f084404358f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:26.016421 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126c184b-90ab-47ae-8294-6f084404358f None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:26.036050 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 187/371] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:29:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:26.036673 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c184b-90ab-47ae-8294-6f084404358f None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:26.111729 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:26.160436 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3f390f-1da2-4439-91a1-17d028841ec3 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:26.209527 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e0c868-944e-4723-adba-445f1acd2f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:26.210142 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e0c868-944e-4723-adba-445f1acd2f04 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:26.210416 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e0c868-944e-4723-adba-445f1acd2f04 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:26.210684 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e0c868-944e-4723-adba-445f1acd2f04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:26.289649 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e0c868-944e-4723-adba-445f1acd2f04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:26.291134 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e0c868-944e-4723-adba-445f1acd2f04 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:26.299586 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:26.306563 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c184b-90ab-47ae-8294-6f084404358f None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:26.379150 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5cbb29-bbb9-456b-80df-2cccd2302a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:26.379782 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5cbb29-bbb9-456b-80df-2cccd2302a26 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:26.380066 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5cbb29-bbb9-456b-80df-2cccd2302a26 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:26.380342 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5cbb29-bbb9-456b-80df-2cccd2302a26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:26.381059 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5cbb29-bbb9-456b-80df-2cccd2302a26 None admin] RBAC: Adding query filter params (role.id=f51dfe78039041e3a947ea27cfe97651, user.id=999b71fa2e67451ba9d3140167051c4f, scope.system=all) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:26.381370 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5cbb29-bbb9-456b-80df-2cccd2302a26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:26.382790 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5cbb29-bbb9-456b-80df-2cccd2302a26 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:26.383920 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:26.400425 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 188/372] 10.222.0.30 () {56 vars in 1308 bytes} [Tue Aug 30 14:29:26 2022] GET /identity/v3/role_assignments?user.id=999b71fa2e67451ba9d3140167051c4f&scope.system=all&role.id=f51dfe78039041e3a947ea27cfe97651 => generated 510 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:26.401267 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5cbb29-bbb9-456b-80df-2cccd2302a26 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:26.482698 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 185/373] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:29:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:29:26.488860 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e0c868-944e-4723-adba-445f1acd2f04 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:26.592923 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5cbb29-bbb9-456b-80df-2cccd2302a26 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:26.644119 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d5a14b-0bf0-4cb1-8dec-7bf05d899086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:26.644743 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d5a14b-0bf0-4cb1-8dec-7bf05d899086 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:26.645020 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d5a14b-0bf0-4cb1-8dec-7bf05d899086 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:26.645287 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d5a14b-0bf0-4cb1-8dec-7bf05d899086 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:26.693419 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:26.725479 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e0c868-944e-4723-adba-445f1acd2f04 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:26.775928 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d5a14b-0bf0-4cb1-8dec-7bf05d899086 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:26.777341 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d5a14b-0bf0-4cb1-8dec-7bf05d899086 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:26.799397 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d3dc55-8f13-41ad-82fd-8d2761bb27c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:26.800049 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3dc55-8f13-41ad-82fd-8d2761bb27c3 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:26.800352 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3dc55-8f13-41ad-82fd-8d2761bb27c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:26.800591 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3dc55-8f13-41ad-82fd-8d2761bb27c3 None admin] PATH_INFO: `/v3/projects/961f8057321f482d9a5d9202c7bc3c4c` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:26.801743 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d3dc55-8f13-41ad-82fd-8d2761bb27c3 None admin] RBAC: Authorizing `identity:get_project(project_id=961f8057321f482d9a5d9202c7bc3c4c)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:26.803217 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d3dc55-8f13-41ad-82fd-8d2761bb27c3 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:26.804838 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 186/374] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:29:26 2022] GET /identity/v3/projects/961f8057321f482d9a5d9202c7bc3c4c => generated 302 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:29:26.811715 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d3dc55-8f13-41ad-82fd-8d2761bb27c3 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:26.869102 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:26.935179 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 189/375] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:29:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:29:26.941498 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d5a14b-0bf0-4cb1-8dec-7bf05d899086 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:26.985475 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d3dc55-8f13-41ad-82fd-8d2761bb27c3 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:27.002571 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:27.044231 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118cf982-2e78-4af3-ad28-1d5c813f5b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:27.044810 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118cf982-2e78-4af3-ad28-1d5c813f5b49 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:27.045087 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118cf982-2e78-4af3-ad28-1d5c813f5b49 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:27.045366 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118cf982-2e78-4af3-ad28-1d5c813f5b49 None admin] PATH_INFO: `/v3/groups/88ad07bcdacd4ef79be074cf74d7f8b2` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:27.046553 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118cf982-2e78-4af3-ad28-1d5c813f5b49 None admin] RBAC: Authorizing `identity:get_group(group_id=88ad07bcdacd4ef79be074cf74d7f8b2)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:27.048013 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118cf982-2e78-4af3-ad28-1d5c813f5b49 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:27.049892 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 187/376] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Aug 30 14:29:26 2022] GET /identity/v3/groups/88ad07bcdacd4ef79be074cf74d7f8b2 => generated 226 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:29:27.050180 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d5a14b-0bf0-4cb1-8dec-7bf05d899086 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:27.055897 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118cf982-2e78-4af3-ad28-1d5c813f5b49 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:27.101899 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211b05e1-b7b4-4307-b363-8d65f34c793a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:27.102562 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211b05e1-b7b4-4307-b363-8d65f34c793a None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:27.102915 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211b05e1-b7b4-4307-b363-8d65f34c793a None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:27.103188 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211b05e1-b7b4-4307-b363-8d65f34c793a None admin] PATH_INFO: `/v3/projects/2ac908dcbf994029b1c39e157d0de0cb` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:27.108128 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211b05e1-b7b4-4307-b363-8d65f34c793a None admin] RBAC: Authorizing `identity:get_project(project_id=2ac908dcbf994029b1c39e157d0de0cb)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:27.109700 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211b05e1-b7b4-4307-b363-8d65f34c793a None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:27.111847 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 190/377] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:29:26 2022] GET /identity/v3/projects/2ac908dcbf994029b1c39e157d0de0cb => generated 344 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:27.117393 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211b05e1-b7b4-4307-b363-8d65f34c793a None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:27.169896 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:27.179829 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118cf982-2e78-4af3-ad28-1d5c813f5b49 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:27.232570 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211b05e1-b7b4-4307-b363-8d65f34c793a None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:27.234670 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b20e85b-25bb-4e54-bf68-10dc1806fbdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:27.235449 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b20e85b-25bb-4e54-bf68-10dc1806fbdf None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:27.235702 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b20e85b-25bb-4e54-bf68-10dc1806fbdf None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:27.235974 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b20e85b-25bb-4e54-bf68-10dc1806fbdf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:27.236707 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b20e85b-25bb-4e54-bf68-10dc1806fbdf None admin] RBAC: Adding query filter params (group.id=88ad07bcdacd4ef79be074cf74d7f8b2, role.id=1c4886a7a7fd402d8221a0158ddcb167, scope.project.id=961f8057321f482d9a5d9202c7bc3c4c) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:27.237012 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b20e85b-25bb-4e54-bf68-10dc1806fbdf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:27.238437 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b20e85b-25bb-4e54-bf68-10dc1806fbdf None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:27.245295 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 188/378] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Aug 30 14:29:27 2022] GET /identity/v3/role_assignments?group.id=88ad07bcdacd4ef79be074cf74d7f8b2&scope.project.id=961f8057321f482d9a5d9202c7bc3c4c&role.id=1c4886a7a7fd402d8221a0158ddcb167 => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:29:27.259773 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:27.283292 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b1f674-7ac8-4840-8f2f-994738155c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:27.283956 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b1f674-7ac8-4840-8f2f-994738155c0c None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:27.284236 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b1f674-7ac8-4840-8f2f-994738155c0c None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:27.284507 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b1f674-7ac8-4840-8f2f-994738155c0c None admin] PATH_INFO: `/v3/groups/9625519eff7243eab5965376fb01cd09` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:27.295313 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b1f674-7ac8-4840-8f2f-994738155c0c None admin] RBAC: Authorizing `identity:get_group(group_id=9625519eff7243eab5965376fb01cd09)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:27.296756 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b1f674-7ac8-4840-8f2f-994738155c0c None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:27.298785 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 191/379] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Aug 30 14:29:27 2022] GET /identity/v3/groups/9625519eff7243eab5965376fb01cd09 => generated 229 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:29:27.308381 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b20e85b-25bb-4e54-bf68-10dc1806fbdf None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:27.443338 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b20e85b-25bb-4e54-bf68-10dc1806fbdf None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:27.479519 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b8980f-3fb8-463b-93d5-4edf599bee53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:27.480178 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b8980f-3fb8-463b-93d5-4edf599bee53 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:27.480499 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b8980f-3fb8-463b-93d5-4edf599bee53 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:27.480892 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b8980f-3fb8-463b-93d5-4edf599bee53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:27.481731 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b8980f-3fb8-463b-93d5-4edf599bee53 None admin] RBAC: Adding query filter params (group.id=9625519eff7243eab5965376fb01cd09, role.id=3cda0615d87d4af0860a2a5e9b4e2c05, scope.project.id=2ac908dcbf994029b1c39e157d0de0cb) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:27.482035 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b8980f-3fb8-463b-93d5-4edf599bee53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:27.483490 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b8980f-3fb8-463b-93d5-4edf599bee53 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:27.489718 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 189/380] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Aug 30 14:29:27 2022] GET /identity/v3/role_assignments?group.id=9625519eff7243eab5965376fb01cd09&scope.project.id=2ac908dcbf994029b1c39e157d0de0cb&role.id=3cda0615d87d4af0860a2a5e9b4e2c05 => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:27.542206 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75b999-efca-4cc7-8806-f88f120149b7 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:27.542854 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75b999-efca-4cc7-8806-f88f120149b7 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:27.542854 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75b999-efca-4cc7-8806-f88f120149b7 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:27.545523 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 192/381] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:27.551371 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea81f2-89ce-41e2-ba66-7b4e6170a9ae None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:27.551656 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea81f2-89ce-41e2-ba66-7b4e6170a9ae None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:27.551969 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea81f2-89ce-41e2-ba66-7b4e6170a9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:27.574309 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-bcea81f2-89ce-41e2-ba66-7b4e6170a9ae None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:27.575665 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-bcea81f2-89ce-41e2-ba66-7b4e6170a9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:27.594910 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:27.658416 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 190/382] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:27.970359 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b75b999-efca-4cc7-8806-f88f120149b7 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:28.069772 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c8273-a285-4991-b6a7-bdfbbf33c9ff None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:28.070097 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c8273-a285-4991-b6a7-bdfbbf33c9ff None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:28.070335 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c8273-a285-4991-b6a7-bdfbbf33c9ff None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:28.074684 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 191/383] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:28.082667 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb333fa9-1d62-48be-aba2-310670902fa6 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:28.082945 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb333fa9-1d62-48be-aba2-310670902fa6 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:28.083253 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb333fa9-1d62-48be-aba2-310670902fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:28.105977 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b75b999-efca-4cc7-8806-f88f120149b7 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:28.115476 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-eb333fa9-1d62-48be-aba2-310670902fa6 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:28.116684 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-eb333fa9-1d62-48be-aba2-310670902fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:28.145960 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba22c23-ad1d-4125-b34d-d6fdb6773313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:28.146936 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba22c23-ad1d-4125-b34d-d6fdb6773313 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:28.147440 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba22c23-ad1d-4125-b34d-d6fdb6773313 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:28.147928 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba22c23-ad1d-4125-b34d-d6fdb6773313 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:28.160772 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba22c23-ad1d-4125-b34d-d6fdb6773313 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:28.166430 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-2ba22c23-ad1d-4125-b34d-d6fdb6773313 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:29:28.166430 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:28.166430 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:28.166430 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:28.166430 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:28.166430 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:28.166430 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:28.166430 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:28.166430 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:28.166430 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:28.166430 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:28.166430 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:28.166430 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:28.166430 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:28.166430 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:28.166430 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:28.167343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:28.167343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:28.167343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:28.167343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:28.167343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:28.167343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:28.167343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:28.167343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:28.167343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:28.167343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:28.167343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:28.167343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:28.167343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:28.167343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:28.167343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:28.167343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:28.168326 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:28.168326 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:28.168326 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:28.168326 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:28.168326 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:28.168326 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:28.168326 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:28.168326 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:28.168326 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:28.168326 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:28.168326 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:29:28.168326 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:28.170685 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 193/384] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:29:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:28.176582 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba22c23-ad1d-4125-b34d-d6fdb6773313 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:28.241993 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:28.254826 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 192/385] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:29:28.256447 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908dc867-0a81-4e52-8e6d-94aab4854dfe None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:28.256937 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908dc867-0a81-4e52-8e6d-94aab4854dfe None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:28.257433 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908dc867-0a81-4e52-8e6d-94aab4854dfe None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:28.258881 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 193/386] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:28.265458 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5245a5-4095-449e-b969-7623d4178c7b None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:28.265985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5245a5-4095-449e-b969-7623d4178c7b None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:28.266459 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5245a5-4095-449e-b969-7623d4178c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:28.289058 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-ec5245a5-4095-449e-b969-7623d4178c7b None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:28.290615 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-ec5245a5-4095-449e-b969-7623d4178c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:28.344986 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba22c23-ad1d-4125-b34d-d6fdb6773313 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:28.380945 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 194/387] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:29:28.415816 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91eaa40-c981-4e4e-a8ec-5e514f49c960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:28.416426 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91eaa40-c981-4e4e-a8ec-5e514f49c960 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:28.416703 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91eaa40-c981-4e4e-a8ec-5e514f49c960 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:28.416968 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91eaa40-c981-4e4e-a8ec-5e514f49c960 None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:28.417675 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91eaa40-c981-4e4e-a8ec-5e514f49c960 None admin] RBAC: Adding query filter params (name=member) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:28.418019 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91eaa40-c981-4e4e-a8ec-5e514f49c960 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:28.419748 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91eaa40-c981-4e4e-a8ec-5e514f49c960 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:28.429154 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 194/388] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 30 14:29:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:28.435279 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5245a5-4095-449e-b969-7623d4178c7b None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:28.453558 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:28.559009 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5245a5-4095-449e-b969-7623d4178c7b None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:28.601982 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd685272-f317-4213-bbb4-9d4f3d31b363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:28.602608 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd685272-f317-4213-bbb4-9d4f3d31b363 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:28.602880 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd685272-f317-4213-bbb4-9d4f3d31b363 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:28.603146 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd685272-f317-4213-bbb4-9d4f3d31b363 None admin] PATH_INFO: `/v3/groups/88ad07bcdacd4ef79be074cf74d7f8b2` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:28.606531 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd685272-f317-4213-bbb4-9d4f3d31b363 None admin] RBAC: Authorizing `identity:get_group(group_id=88ad07bcdacd4ef79be074cf74d7f8b2)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:28.608053 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd685272-f317-4213-bbb4-9d4f3d31b363 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:28.609786 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 195/389] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Aug 30 14:29:28 2022] GET /identity/v3/groups/88ad07bcdacd4ef79be074cf74d7f8b2 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:29:28.615298 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91eaa40-c981-4e4e-a8ec-5e514f49c960 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:28.620501 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd685272-f317-4213-bbb4-9d4f3d31b363 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:28.767408 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91eaa40-c981-4e4e-a8ec-5e514f49c960 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:28.769436 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:28.783157 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:28.798998 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd685272-f317-4213-bbb4-9d4f3d31b363 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:28.817290 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f0e79-5b78-4ed8-8efe-b1c433d1b543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:28.817987 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f0e79-5b78-4ed8-8efe-b1c433d1b543 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:28.818271 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f0e79-5b78-4ed8-8efe-b1c433d1b543 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:28.818548 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f0e79-5b78-4ed8-8efe-b1c433d1b543 None admin] PATH_INFO: `/v3/projects/1c2139beef6b44b9a0574047298c1af9` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:28.820036 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6f0e79-5b78-4ed8-8efe-b1c433d1b543 None admin] RBAC: Authorizing `identity:get_project(project_id=1c2139beef6b44b9a0574047298c1af9)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:28.821902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6f0e79-5b78-4ed8-8efe-b1c433d1b543 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:28.824033 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 195/390] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:29:28 2022] GET /identity/v3/projects/1c2139beef6b44b9a0574047298c1af9 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:28.831637 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f0e79-5b78-4ed8-8efe-b1c433d1b543 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:28.846394 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f60bd3-a6e8-4ab6-8c17-70fe7c1bf3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:28.847806 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f60bd3-a6e8-4ab6-8c17-70fe7c1bf3c9 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:28.848725 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f60bd3-a6e8-4ab6-8c17-70fe7c1bf3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:28.849213 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f60bd3-a6e8-4ab6-8c17-70fe7c1bf3c9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:28.866063 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f60bd3-a6e8-4ab6-8c17-70fe7c1bf3c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:28.874899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-94f60bd3-a6e8-4ab6-8c17-70fe7c1bf3c9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:29:28.874899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:28.874899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:28.874899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:28.874899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:28.874899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:28.874899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:28.874899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:28.874899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:28.874899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:28.874899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:28.874899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:28.874899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:28.874899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:28.874899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:28.874899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:28.875999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:28.875999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:28.875999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:28.875999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:28.875999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:28.875999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:28.875999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:28.875999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:28.875999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:28.875999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:28.875999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:28.875999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:28.875999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:28.875999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:28.875999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:28.875999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:28.876796 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:28.876796 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:28.876796 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:28.876796 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:28.876796 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:28.876796 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:28.876796 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:28.876796 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:28.876796 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:28.876796 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:28.876796 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:29:28.876796 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:28.880407 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 196/391] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 30 14:29:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:29:28.881379 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f60bd3-a6e8-4ab6-8c17-70fe7c1bf3c9 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:28.953713 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f0e79-5b78-4ed8-8efe-b1c433d1b543 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:28.989267 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99326f57-b2d0-4aa6-9c95-7a7c3cadb00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:28.990209 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99326f57-b2d0-4aa6-9c95-7a7c3cadb00e None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:28.990645 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f60bd3-a6e8-4ab6-8c17-70fe7c1bf3c9 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:28.990923 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99326f57-b2d0-4aa6-9c95-7a7c3cadb00e None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:28.991421 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99326f57-b2d0-4aa6-9c95-7a7c3cadb00e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:29.002859 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99326f57-b2d0-4aa6-9c95-7a7c3cadb00e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:29.009250 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-99326f57-b2d0-4aa6-9c95-7a7c3cadb00e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:29:29.009250 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:29.009250 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:29.009250 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:29.009250 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:29.009250 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:29.009250 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:29.009250 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:29.009250 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:29.009250 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:29.009250 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:29.009250 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:29.009250 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:29.009250 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:29.009250 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:29.009250 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:29.010149 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:29.010149 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:29.010149 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:29.010149 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:29.010149 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:29.010149 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:29.010149 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:29.010149 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:29.010149 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:29.010149 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:29.010149 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:29.010149 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:29.010149 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:29.010149 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:29.010149 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:29.010149 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:29.011021 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:29.011021 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:29.011021 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:29.011021 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:29.011021 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:29.011021 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:29.011021 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:29.011021 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:29.011021 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:29.011021 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:29.011021 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:29:29.011021 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:29.013602 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 196/392] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:29:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:29.019239 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99326f57-b2d0-4aa6-9c95-7a7c3cadb00e None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:29.026518 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea52a2e-3fef-4f35-87f2-f0d2b5569396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:29.027192 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea52a2e-3fef-4f35-87f2-f0d2b5569396 None admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:29.027708 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea52a2e-3fef-4f35-87f2-f0d2b5569396 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:29.027989 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea52a2e-3fef-4f35-87f2-f0d2b5569396 None admin] PATH_INFO: `/v3/projects/1c2139beef6b44b9a0574047298c1af9/groups/88ad07bcdacd4ef79be074cf74d7f8b2/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:29.029978 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea52a2e-3fef-4f35-87f2-f0d2b5569396 None admin] RBAC: Authorizing `identity:create_grant(project_id=1c2139beef6b44b9a0574047298c1af9, group_id=88ad07bcdacd4ef79be074cf74d7f8b2, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:29.031471 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea52a2e-3fef-4f35-87f2-f0d2b5569396 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:29.041507 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:29.049154 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 197/393] 10.222.0.30 () {58 vars in 1442 bytes} [Tue Aug 30 14:29:28 2022] PUT /identity/v3/projects/1c2139beef6b44b9a0574047298c1af9/groups/88ad07bcdacd4ef79be074cf74d7f8b2/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:29:29.050180 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea52a2e-3fef-4f35-87f2-f0d2b5569396 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:29.067141 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:29.172394 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99326f57-b2d0-4aa6-9c95-7a7c3cadb00e None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:29.190432 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea52a2e-3fef-4f35-87f2-f0d2b5569396 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:29.233580 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33672e28-eed6-490d-a210-57f1cf5144e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:29.234320 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33672e28-eed6-490d-a210-57f1cf5144e1 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:29.234603 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33672e28-eed6-490d-a210-57f1cf5144e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:29.234877 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33672e28-eed6-490d-a210-57f1cf5144e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:29.235583 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33672e28-eed6-490d-a210-57f1cf5144e1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:29.235939 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33672e28-eed6-490d-a210-57f1cf5144e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:29.237327 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33672e28-eed6-490d-a210-57f1cf5144e1 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:29.252914 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 197/394] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:29:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:29.264301 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33672e28-eed6-490d-a210-57f1cf5144e1 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:29.265205 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b0fdf-03dd-45c8-81cf-c1fe9e4b6517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:29.266024 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b0fdf-03dd-45c8-81cf-c1fe9e4b6517 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:29.266447 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b0fdf-03dd-45c8-81cf-c1fe9e4b6517 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:29.266894 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b0fdf-03dd-45c8-81cf-c1fe9e4b6517 None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:29.267910 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62b0fdf-03dd-45c8-81cf-c1fe9e4b6517 None admin] RBAC: Adding query filter params (name=member) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:29.268348 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62b0fdf-03dd-45c8-81cf-c1fe9e4b6517 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:29.270062 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62b0fdf-03dd-45c8-81cf-c1fe9e4b6517 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:29.281042 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 198/395] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 30 14:29:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:29.287178 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b0fdf-03dd-45c8-81cf-c1fe9e4b6517 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:29.362537 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:29.408233 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b0fdf-03dd-45c8-81cf-c1fe9e4b6517 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:29.420084 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:29.424513 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33672e28-eed6-490d-a210-57f1cf5144e1 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:29.465663 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcf9e72-5f78-4b22-a8d5-de0b8b0be3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:29.467092 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcf9e72-5f78-4b22-a8d5-de0b8b0be3aa None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:29.467667 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcf9e72-5f78-4b22-a8d5-de0b8b0be3aa None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:29.468212 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcf9e72-5f78-4b22-a8d5-de0b8b0be3aa None admin] PATH_INFO: `/v3/groups/88ad07bcdacd4ef79be074cf74d7f8b2` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:29.469644 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcf9e72-5f78-4b22-a8d5-de0b8b0be3aa None admin] RBAC: Authorizing `identity:get_group(group_id=88ad07bcdacd4ef79be074cf74d7f8b2)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:29.471457 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcf9e72-5f78-4b22-a8d5-de0b8b0be3aa None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:29.473912 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 198/396] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Aug 30 14:29:29 2022] GET /identity/v3/groups/88ad07bcdacd4ef79be074cf74d7f8b2 => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:29.480976 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcf9e72-5f78-4b22-a8d5-de0b8b0be3aa None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:29.495911 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c890f8-32cb-4f9e-8ece-49ae92bc17e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:29.496624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c890f8-32cb-4f9e-8ece-49ae92bc17e0 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:29.496900 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c890f8-32cb-4f9e-8ece-49ae92bc17e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:29.497169 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c890f8-32cb-4f9e-8ece-49ae92bc17e0 None admin] PATH_INFO: `/v3/groups/88ad07bcdacd4ef79be074cf74d7f8b2` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:29.498322 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c890f8-32cb-4f9e-8ece-49ae92bc17e0 None admin] RBAC: Authorizing `identity:get_group(group_id=88ad07bcdacd4ef79be074cf74d7f8b2)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:29.499782 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c890f8-32cb-4f9e-8ece-49ae92bc17e0 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:29.501639 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 199/397] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Aug 30 14:29:29 2022] GET /identity/v3/groups/88ad07bcdacd4ef79be074cf74d7f8b2 => generated 226 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:29.506888 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c890f8-32cb-4f9e-8ece-49ae92bc17e0 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:29.635169 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c890f8-32cb-4f9e-8ece-49ae92bc17e0 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:29.675038 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcf9e72-5f78-4b22-a8d5-de0b8b0be3aa None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:29.689241 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:29.703245 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab4b0dc-36a9-45e1-aa9f-679a1e3c6525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:29.704200 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4b0dc-36a9-45e1-aa9f-679a1e3c6525 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:29.704680 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4b0dc-36a9-45e1-aa9f-679a1e3c6525 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:29.705163 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4b0dc-36a9-45e1-aa9f-679a1e3c6525 None admin] PATH_INFO: `/v3/projects/961f8057321f482d9a5d9202c7bc3c4c` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:29.711946 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab4b0dc-36a9-45e1-aa9f-679a1e3c6525 None admin] RBAC: Authorizing `identity:get_project(project_id=961f8057321f482d9a5d9202c7bc3c4c)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:29.713649 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab4b0dc-36a9-45e1-aa9f-679a1e3c6525 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:29.715840 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 200/398] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:29:29 2022] GET /identity/v3/projects/961f8057321f482d9a5d9202c7bc3c4c => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:29:29.719890 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da0316e-8860-4543-a66f-e4340a92f182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:29.720848 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0316e-8860-4543-a66f-e4340a92f182 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:29.721214 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab4b0dc-36a9-45e1-aa9f-679a1e3c6525 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:29.721569 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0316e-8860-4543-a66f-e4340a92f182 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:29.722040 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0316e-8860-4543-a66f-e4340a92f182 None admin] PATH_INFO: `/v3/projects/1c2139beef6b44b9a0574047298c1af9` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:29.723413 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da0316e-8860-4543-a66f-e4340a92f182 None admin] RBAC: Authorizing `identity:get_project(project_id=1c2139beef6b44b9a0574047298c1af9)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:29.725121 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da0316e-8860-4543-a66f-e4340a92f182 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:29.727407 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 199/399] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:29:29 2022] GET /identity/v3/projects/1c2139beef6b44b9a0574047298c1af9 => generated 298 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:29.736851 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba51a58-8dfd-42a8-b27c-c5486d5656de None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:29.737430 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba51a58-8dfd-42a8-b27c-c5486d5656de None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:29.737990 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba51a58-8dfd-42a8-b27c-c5486d5656de None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:29.743812 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 200/400] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:29 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:29.744887 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba51a58-8dfd-42a8-b27c-c5486d5656de None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:29.784922 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:29.847034 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab4b0dc-36a9-45e1-aa9f-679a1e3c6525 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:29.859355 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba51a58-8dfd-42a8-b27c-c5486d5656de None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:29.884808 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7fab00-c470-4ffa-b402-8ec56c2b7285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:29.885470 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7fab00-c470-4ffa-b402-8ec56c2b7285 None admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:29.885871 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7fab00-c470-4ffa-b402-8ec56c2b7285 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:29.886149 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7fab00-c470-4ffa-b402-8ec56c2b7285 None admin] PATH_INFO: `/v3/projects/961f8057321f482d9a5d9202c7bc3c4c/groups/88ad07bcdacd4ef79be074cf74d7f8b2/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:29.888478 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7fab00-c470-4ffa-b402-8ec56c2b7285 None admin] RBAC: Authorizing `identity:create_grant(project_id=961f8057321f482d9a5d9202c7bc3c4c, group_id=88ad07bcdacd4ef79be074cf74d7f8b2, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:29.890149 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7fab00-c470-4ffa-b402-8ec56c2b7285 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:29.894669 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa8b69e-ea5d-4b9f-8fe1-3abfb775ff20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:29.895367 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa8b69e-ea5d-4b9f-8fe1-3abfb775ff20 None admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:29.895648 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa8b69e-ea5d-4b9f-8fe1-3abfb775ff20 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:29.895924 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa8b69e-ea5d-4b9f-8fe1-3abfb775ff20 None admin] PATH_INFO: `/v3/projects/1c2139beef6b44b9a0574047298c1af9/groups/88ad07bcdacd4ef79be074cf74d7f8b2/roles/1c4886a7a7fd402d8221a0158ddcb167` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:29.898336 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa8b69e-ea5d-4b9f-8fe1-3abfb775ff20 None admin] RBAC: Authorizing `identity:create_grant(project_id=1c2139beef6b44b9a0574047298c1af9, group_id=88ad07bcdacd4ef79be074cf74d7f8b2, role_id=1c4886a7a7fd402d8221a0158ddcb167)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:29.899812 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa8b69e-ea5d-4b9f-8fe1-3abfb775ff20 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:29.903686 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 201/401] 10.222.0.30 () {58 vars in 1442 bytes} [Tue Aug 30 14:29:29 2022] PUT /identity/v3/projects/961f8057321f482d9a5d9202c7bc3c4c/groups/88ad07bcdacd4ef79be074cf74d7f8b2/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:29:29.905178 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395bc5b-4a8f-4342-b741-7abe5d2325eb None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:29.905519 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395bc5b-4a8f-4342-b741-7abe5d2325eb None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:29.905800 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395bc5b-4a8f-4342-b741-7abe5d2325eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:29.923674 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 201/402] 10.222.0.30 () {58 vars in 1442 bytes} [Tue Aug 30 14:29:29 2022] PUT /identity/v3/projects/1c2139beef6b44b9a0574047298c1af9/groups/88ad07bcdacd4ef79be074cf74d7f8b2/roles/1c4886a7a7fd402d8221a0158ddcb167 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:29:29.925493 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81adc4e1-ab74-41b0-a64d-0106da5f7e6c None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:29.925968 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81adc4e1-ab74-41b0-a64d-0106da5f7e6c None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:29.926421 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81adc4e1-ab74-41b0-a64d-0106da5f7e6c None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:29.927759 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-6395bc5b-4a8f-4342-b741-7abe5d2325eb None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:29.928871 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 202/403] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:29.929332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-6395bc5b-4a8f-4342-b741-7abe5d2325eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:29.944821 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6c93e-cdcb-4a1f-8993-295f3b280777 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:29.945407 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6c93e-cdcb-4a1f-8993-295f3b280777 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:29.945863 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6c93e-cdcb-4a1f-8993-295f3b280777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:29.985996 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-9ee6c93e-cdcb-4a1f-8993-295f3b280777 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:29.987692 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-9ee6c93e-cdcb-4a1f-8993-295f3b280777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:30.045596 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:30.065639 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 202/404] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:30.099573 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 203/405] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:30.355013 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6395bc5b-4a8f-4342-b741-7abe5d2325eb None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:30.479511 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6395bc5b-4a8f-4342-b741-7abe5d2325eb None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:30.495436 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee6c93e-cdcb-4a1f-8993-295f3b280777 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:30.524461 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371df3ed-169c-4271-8053-f0e9f4774476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:30.525109 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371df3ed-169c-4271-8053-f0e9f4774476 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:30.525633 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371df3ed-169c-4271-8053-f0e9f4774476 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:30.525907 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371df3ed-169c-4271-8053-f0e9f4774476 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:30.549120 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:30.549532 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371df3ed-169c-4271-8053-f0e9f4774476 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:30.554599 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-371df3ed-169c-4271-8053-f0e9f4774476 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:29:30.554599 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:30.554599 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:30.554599 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:30.554599 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:30.554599 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:30.554599 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:30.554599 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:30.554599 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:30.554599 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:30.554599 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:30.554599 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:30.554599 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:30.554599 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:30.554599 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:30.554599 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:30.556247 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:30.556247 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:30.556247 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:30.556247 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:30.556247 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:30.556247 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:30.556247 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:30.556247 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:30.556247 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:30.556247 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:30.556247 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:30.556247 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:30.556247 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:30.556247 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:30.556247 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:30.556247 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:30.557106 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:30.557106 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:30.557106 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:30.557106 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:30.557106 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:30.557106 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:30.557106 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:30.557106 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:30.557106 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:30.557106 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:30.557106 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:29:30.557106 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:30.563231 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 203/406] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 30 14:29:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:30.564413 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371df3ed-169c-4271-8053-f0e9f4774476 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:30.646369 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:30.657526 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee6c93e-cdcb-4a1f-8993-295f3b280777 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:30.707559 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371df3ed-169c-4271-8053-f0e9f4774476 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:30.727404 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f8e0c6-a130-47fb-a4f8-c7d27b90c345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:30.728850 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8e0c6-a130-47fb-a4f8-c7d27b90c345 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:30.729378 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8e0c6-a130-47fb-a4f8-c7d27b90c345 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:30.729949 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8e0c6-a130-47fb-a4f8-c7d27b90c345 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:30.745591 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd5d64b-9acc-411b-89ba-3447cb953cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:30.746224 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd5d64b-9acc-411b-89ba-3447cb953cf0 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:30.746501 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd5d64b-9acc-411b-89ba-3447cb953cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:30.746774 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd5d64b-9acc-411b-89ba-3447cb953cf0 None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:30.747841 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd5d64b-9acc-411b-89ba-3447cb953cf0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:30.748144 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd5d64b-9acc-411b-89ba-3447cb953cf0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:30.749893 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd5d64b-9acc-411b-89ba-3447cb953cf0 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:30.755836 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f8e0c6-a130-47fb-a4f8-c7d27b90c345 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:30.758381 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 204/407] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:29:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:29:30.761187 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-91f8e0c6-a130-47fb-a4f8-c7d27b90c345 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:29:30.761187 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:30.761187 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:30.761187 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:30.761187 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:30.761187 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:30.761187 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:30.761187 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:30.761187 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:30.761187 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:30.761187 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:30.761187 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:30.761187 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:30.761187 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:30.761187 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:30.761187 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:30.762848 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:30.762848 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:30.762848 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:30.762848 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:30.762848 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:30.762848 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:30.762848 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:30.762848 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:30.762848 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:30.762848 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:30.762848 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:30.762848 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:30.762848 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:30.762848 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:30.762848 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:30.762848 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:30.763670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:30.763670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:30.763670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:30.763670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:30.763670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:30.763670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:30.763670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:30.763670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:30.763670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:30.763670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:30.763670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:29:30.763670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:30.769659 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd5d64b-9acc-411b-89ba-3447cb953cf0 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:30.770245 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 204/408] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 30 14:29:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:30.776333 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f8e0c6-a130-47fb-a4f8-c7d27b90c345 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:30.884784 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd5d64b-9acc-411b-89ba-3447cb953cf0 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:30.896116 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:30.924442 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a9788d-3803-48ae-a695-5c9e71cc273b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:30.925081 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a9788d-3803-48ae-a695-5c9e71cc273b None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:30.925357 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a9788d-3803-48ae-a695-5c9e71cc273b None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:30.925627 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a9788d-3803-48ae-a695-5c9e71cc273b None admin] PATH_INFO: `/v3/groups/88ad07bcdacd4ef79be074cf74d7f8b2` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:30.931932 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a9788d-3803-48ae-a695-5c9e71cc273b None admin] RBAC: Authorizing `identity:get_group(group_id=88ad07bcdacd4ef79be074cf74d7f8b2)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:30.932056 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:30.933950 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a9788d-3803-48ae-a695-5c9e71cc273b None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:30.936062 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 205/409] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Aug 30 14:29:30 2022] GET /identity/v3/groups/88ad07bcdacd4ef79be074cf74d7f8b2 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:29:30.941916 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a9788d-3803-48ae-a695-5c9e71cc273b None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:30.955411 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f8e0c6-a130-47fb-a4f8-c7d27b90c345 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:31.032024 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc996609-fb53-45d2-ae7f-f9dc7372ff82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:31.032545 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc996609-fb53-45d2-ae7f-f9dc7372ff82 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:31.032828 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc996609-fb53-45d2-ae7f-f9dc7372ff82 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:31.033102 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc996609-fb53-45d2-ae7f-f9dc7372ff82 None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:31.033775 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc996609-fb53-45d2-ae7f-f9dc7372ff82 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:31.034069 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc996609-fb53-45d2-ae7f-f9dc7372ff82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:31.035483 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc996609-fb53-45d2-ae7f-f9dc7372ff82 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:31.057179 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 205/410] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:29:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:29:31.068188 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc996609-fb53-45d2-ae7f-f9dc7372ff82 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:31.080818 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a9788d-3803-48ae-a695-5c9e71cc273b None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:31.116046 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c7086-f6bf-419e-99e2-cbe5caf8777f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:31.117084 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c7086-f6bf-419e-99e2-cbe5caf8777f None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:31.117582 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c7086-f6bf-419e-99e2-cbe5caf8777f None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:31.118061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c7086-f6bf-419e-99e2-cbe5caf8777f None admin] PATH_INFO: `/v3/projects/961f8057321f482d9a5d9202c7bc3c4c` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:31.119445 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227c7086-f6bf-419e-99e2-cbe5caf8777f None admin] RBAC: Authorizing `identity:get_project(project_id=961f8057321f482d9a5d9202c7bc3c4c)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:31.121691 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227c7086-f6bf-419e-99e2-cbe5caf8777f None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:31.123936 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 206/411] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:29:30 2022] GET /identity/v3/projects/961f8057321f482d9a5d9202c7bc3c4c => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:29:31.129106 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c7086-f6bf-419e-99e2-cbe5caf8777f None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:31.207043 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:31.267421 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c7086-f6bf-419e-99e2-cbe5caf8777f None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:31.275973 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc996609-fb53-45d2-ae7f-f9dc7372ff82 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:31.305203 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc388b5-e095-49b4-bcdf-33216de465a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:31.306205 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc388b5-e095-49b4-bcdf-33216de465a1 None admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:31.306765 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc388b5-e095-49b4-bcdf-33216de465a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:31.307387 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc388b5-e095-49b4-bcdf-33216de465a1 None admin] PATH_INFO: `/v3/projects/961f8057321f482d9a5d9202c7bc3c4c/groups/88ad07bcdacd4ef79be074cf74d7f8b2/roles/1c4886a7a7fd402d8221a0158ddcb167` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:31.310824 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc388b5-e095-49b4-bcdf-33216de465a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=961f8057321f482d9a5d9202c7bc3c4c, group_id=88ad07bcdacd4ef79be074cf74d7f8b2, role_id=1c4886a7a7fd402d8221a0158ddcb167)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:31.312594 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc388b5-e095-49b4-bcdf-33216de465a1 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:31.319478 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4718d7-ecaa-4bb0-8514-840a54508b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:31.320220 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4718d7-ecaa-4bb0-8514-840a54508b32 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:31.320509 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4718d7-ecaa-4bb0-8514-840a54508b32 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:31.320793 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4718d7-ecaa-4bb0-8514-840a54508b32 None admin] PATH_INFO: `/v3/groups/9625519eff7243eab5965376fb01cd09` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:31.324117 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4718d7-ecaa-4bb0-8514-840a54508b32 None admin] RBAC: Authorizing `identity:get_group(group_id=9625519eff7243eab5965376fb01cd09)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:31.325774 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4718d7-ecaa-4bb0-8514-840a54508b32 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:31.327706 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 206/412] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Aug 30 14:29:31 2022] GET /identity/v3/groups/9625519eff7243eab5965376fb01cd09 => generated 229 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:29:31.329339 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 207/413] 10.222.0.30 () {58 vars in 1442 bytes} [Tue Aug 30 14:29:31 2022] PUT /identity/v3/projects/961f8057321f482d9a5d9202c7bc3c4c/groups/88ad07bcdacd4ef79be074cf74d7f8b2/roles/1c4886a7a7fd402d8221a0158ddcb167 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:29:31.343990 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4718d7-ecaa-4bb0-8514-840a54508b32 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:31.381826 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:31.514230 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4718d7-ecaa-4bb0-8514-840a54508b32 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:31.561839 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61121d3c-79aa-4a86-9624-77ba208b50d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:31.562849 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61121d3c-79aa-4a86-9624-77ba208b50d3 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:31.563366 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61121d3c-79aa-4a86-9624-77ba208b50d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:31.563903 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61121d3c-79aa-4a86-9624-77ba208b50d3 None admin] PATH_INFO: `/v3/projects/2ac908dcbf994029b1c39e157d0de0cb` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:31.565251 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61121d3c-79aa-4a86-9624-77ba208b50d3 None admin] RBAC: Authorizing `identity:get_project(project_id=2ac908dcbf994029b1c39e157d0de0cb)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:31.566892 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61121d3c-79aa-4a86-9624-77ba208b50d3 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:31.568773 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 207/414] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:29:31 2022] GET /identity/v3/projects/2ac908dcbf994029b1c39e157d0de0cb => generated 344 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:31.575312 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc388b5-e095-49b4-bcdf-33216de465a1 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:31.622256 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a0bff-1f1d-40c2-b6e5-2f76d0dcfc41 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:31.622913 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a0bff-1f1d-40c2-b6e5-2f76d0dcfc41 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:31.623572 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a0bff-1f1d-40c2-b6e5-2f76d0dcfc41 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:31.625718 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 208/415] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:31.635190 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c1b8b-fb04-49a0-9af7-325ec418d719 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:31.635771 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c1b8b-fb04-49a0-9af7-325ec418d719 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:31.636263 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c1b8b-fb04-49a0-9af7-325ec418d719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:31.665866 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-991c1b8b-fb04-49a0-9af7-325ec418d719 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:31.669187 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-991c1b8b-fb04-49a0-9af7-325ec418d719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:31.765339 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc388b5-e095-49b4-bcdf-33216de465a1 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:31.781531 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:31.815659 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 209/416] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:29:31.840338 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a08ce15-6b67-42b5-8f46-64fe2e221223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:31.841176 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08ce15-6b67-42b5-8f46-64fe2e221223 None admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:31.841468 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08ce15-6b67-42b5-8f46-64fe2e221223 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:31.841743 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08ce15-6b67-42b5-8f46-64fe2e221223 None admin] PATH_INFO: `/v3/projects/2ac908dcbf994029b1c39e157d0de0cb/groups/9625519eff7243eab5965376fb01cd09/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:31.843792 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a08ce15-6b67-42b5-8f46-64fe2e221223 None admin] RBAC: Authorizing `identity:create_grant(project_id=2ac908dcbf994029b1c39e157d0de0cb, group_id=9625519eff7243eab5965376fb01cd09, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:31.845244 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a08ce15-6b67-42b5-8f46-64fe2e221223 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:31.865604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 208/417] 10.222.0.30 () {58 vars in 1442 bytes} [Tue Aug 30 14:29:31 2022] PUT /identity/v3/projects/2ac908dcbf994029b1c39e157d0de0cb/groups/9625519eff7243eab5965376fb01cd09/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:29:32.127048 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991c1b8b-fb04-49a0-9af7-325ec418d719 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:32.142986 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:32.159135 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ddf75-46c7-4675-bb3f-04afa867f978 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:32.159619 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ddf75-46c7-4675-bb3f-04afa867f978 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:32.159903 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ddf75-46c7-4675-bb3f-04afa867f978 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:32.162166 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 209/418] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:32.167168 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d42ae5f-3935-4518-b125-dcd9c35257e6 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:32.167483 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d42ae5f-3935-4518-b125-dcd9c35257e6 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:32.168290 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d42ae5f-3935-4518-b125-dcd9c35257e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:32.191971 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-9d42ae5f-3935-4518-b125-dcd9c35257e6 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:32.193165 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-9d42ae5f-3935-4518-b125-dcd9c35257e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:32.251064 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991c1b8b-fb04-49a0-9af7-325ec418d719 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:32.298396 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e0e742-e765-4cba-956c-5bd38965c0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:32.299471 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e0e742-e765-4cba-956c-5bd38965c0d0 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:32.299971 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e0e742-e765-4cba-956c-5bd38965c0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:32.300474 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e0e742-e765-4cba-956c-5bd38965c0d0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:32.315873 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 210/419] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:29:32.320510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e0e742-e765-4cba-956c-5bd38965c0d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:32.326358 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-10e0e742-e765-4cba-956c-5bd38965c0d0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:29:32.326358 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:32.326358 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:32.326358 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:32.326358 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:32.326358 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:32.326358 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:32.326358 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:32.326358 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:32.326358 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:32.326358 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:32.326358 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:32.326358 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:32.326358 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:32.326358 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:32.326358 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:32.327290 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:32.327290 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:32.327290 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:32.327290 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:32.327290 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:32.327290 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:32.327290 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:32.327290 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:32.327290 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:32.327290 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:32.327290 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:32.327290 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:32.327290 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:32.327290 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:32.327290 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:32.327290 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:32.328100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:32.328100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:32.328100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:32.328100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:32.328100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:32.328100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:32.328100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:32.328100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:32.328100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:32.328100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:32.328100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:29:32.328100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:32.328100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 210/420] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:29:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:32.339449 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d42ae5f-3935-4518-b125-dcd9c35257e6 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:32.467510 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:32.479456 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d42ae5f-3935-4518-b125-dcd9c35257e6 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:32.521215 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17496435-bd8a-4993-9863-58cbc0a867e7 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:32.521536 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17496435-bd8a-4993-9863-58cbc0a867e7 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:32.521767 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17496435-bd8a-4993-9863-58cbc0a867e7 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:32.524924 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 211/421] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:29:32.529060 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a91d7d-65fe-4fbe-9044-b583fe164397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:32.529680 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a91d7d-65fe-4fbe-9044-b583fe164397 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:32.529962 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a91d7d-65fe-4fbe-9044-b583fe164397 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:32.530060 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f332ae40-be92-4b4d-9e56-973078c7ba08 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:32.530335 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a91d7d-65fe-4fbe-9044-b583fe164397 None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:32.530441 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f332ae40-be92-4b4d-9e56-973078c7ba08 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:32.530634 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f332ae40-be92-4b4d-9e56-973078c7ba08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:32.530956 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a91d7d-65fe-4fbe-9044-b583fe164397 None admin] RBAC: Adding query filter params (name=member) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:32.531286 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a91d7d-65fe-4fbe-9044-b583fe164397 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:32.532662 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a91d7d-65fe-4fbe-9044-b583fe164397 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:32.543368 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 211/422] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 30 14:29:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:32.552189 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a91d7d-65fe-4fbe-9044-b583fe164397 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:32.579831 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-f332ae40-be92-4b4d-9e56-973078c7ba08 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:32.581631 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-f332ae40-be92-4b4d-9e56-973078c7ba08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:32.671626 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 212/423] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:29:32.679150 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f332ae40-be92-4b4d-9e56-973078c7ba08 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:32.682960 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:32.784452 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a91d7d-65fe-4fbe-9044-b583fe164397 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:32.805397 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f332ae40-be92-4b4d-9e56-973078c7ba08 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:32.865857 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0094f3c-1092-48fd-a826-3c80566616df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:32.866470 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0094f3c-1092-48fd-a826-3c80566616df None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:32.866748 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0094f3c-1092-48fd-a826-3c80566616df None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:32.867016 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0094f3c-1092-48fd-a826-3c80566616df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:32.874450 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d9a822-2d4d-48f0-a5d9-61f8f7cb9c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:32.875078 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d9a822-2d4d-48f0-a5d9-61f8f7cb9c76 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:32.875383 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d9a822-2d4d-48f0-a5d9-61f8f7cb9c76 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:32.875653 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d9a822-2d4d-48f0-a5d9-61f8f7cb9c76 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:32.894590 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d9a822-2d4d-48f0-a5d9-61f8f7cb9c76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:32.899358 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-26d9a822-2d4d-48f0-a5d9-61f8f7cb9c76 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:29:32.899358 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:32.899358 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:32.899358 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:32.899358 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:32.899358 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:32.899358 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:32.899358 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:32.899358 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:32.899358 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:32.899358 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:32.899358 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:32.899358 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:32.899358 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:32.899358 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:32.899358 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:32.900247 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:32.900247 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:32.900247 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:32.900247 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:32.900247 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:32.900247 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:32.900247 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:32.900247 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:32.900247 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:32.900247 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:32.900247 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:32.900247 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:32.900247 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:32.900247 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:32.900247 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:32.900247 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:32.901263 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:32.901263 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:32.901263 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:32.901263 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:32.901263 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:32.901263 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:32.901263 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:32.901263 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:32.901263 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:32.901263 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:32.901263 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:29:32.901263 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:32.901263 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 213/424] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 30 14:29:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:29:32.908112 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d9a822-2d4d-48f0-a5d9-61f8f7cb9c76 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:32.963606 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:32.968083 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0094f3c-1092-48fd-a826-3c80566616df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:32.970464 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0094f3c-1092-48fd-a826-3c80566616df None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:33.036527 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:33.083586 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 212/425] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:29:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:29:33.084549 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0094f3c-1092-48fd-a826-3c80566616df None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:33.090345 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d9a822-2d4d-48f0-a5d9-61f8f7cb9c76 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:33.135329 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bda6582-495a-4009-88f5-6d11c489cb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:33.135752 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda6582-495a-4009-88f5-6d11c489cb67 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:33.136031 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda6582-495a-4009-88f5-6d11c489cb67 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:33.136308 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda6582-495a-4009-88f5-6d11c489cb67 None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:33.137121 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bda6582-495a-4009-88f5-6d11c489cb67 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:33.137423 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bda6582-495a-4009-88f5-6d11c489cb67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:33.138887 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bda6582-495a-4009-88f5-6d11c489cb67 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:33.148359 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 214/426] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:29:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:33.148710 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bda6582-495a-4009-88f5-6d11c489cb67 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:33.166895 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:33.220126 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0094f3c-1092-48fd-a826-3c80566616df None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:33.258808 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f10d9-d2a9-4d5f-bd9b-ae9d61d25e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:33.260081 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f10d9-d2a9-4d5f-bd9b-ae9d61d25e9a None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:33.260796 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f10d9-d2a9-4d5f-bd9b-ae9d61d25e9a None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:33.261222 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f10d9-d2a9-4d5f-bd9b-ae9d61d25e9a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:33.277785 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9f10d9-d2a9-4d5f-bd9b-ae9d61d25e9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:33.284805 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-9f9f10d9-d2a9-4d5f-bd9b-ae9d61d25e9a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:29:33.284805 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:33.284805 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:33.284805 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:33.284805 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:33.284805 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:33.284805 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:33.284805 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:33.284805 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:33.284805 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:33.284805 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:33.284805 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:33.284805 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:33.284805 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:33.284805 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:33.284805 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:33.285916 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:33.285916 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:33.285916 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:33.285916 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:33.285916 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:33.285916 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:33.285916 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:33.285916 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:33.285916 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:33.285916 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:33.285916 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:33.285916 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:33.285916 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:33.285916 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:33.285916 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:33.285916 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:33.286941 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:33.286941 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:33.286941 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:33.286941 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:33.286941 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:33.286941 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:33.286941 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:33.286941 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:33.286941 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:33.286941 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:33.286941 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 14:29:33.286941 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:33.294919 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bda6582-495a-4009-88f5-6d11c489cb67 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:33.295113 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 213/427] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:29:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:33.297008 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f10d9-d2a9-4d5f-bd9b-ae9d61d25e9a None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:33.326270 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:33.339425 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a751a-23cc-4ffc-a753-e5180605bc35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:33.340892 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a751a-23cc-4ffc-a753-e5180605bc35 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:33.341275 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a751a-23cc-4ffc-a753-e5180605bc35 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:33.341651 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a751a-23cc-4ffc-a753-e5180605bc35 None admin] PATH_INFO: `/v3/projects/1c2139beef6b44b9a0574047298c1af9` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:33.343138 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8a751a-23cc-4ffc-a753-e5180605bc35 None admin] RBAC: Authorizing `identity:get_project(project_id=1c2139beef6b44b9a0574047298c1af9)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:33.345642 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8a751a-23cc-4ffc-a753-e5180605bc35 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:33.347796 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 215/428] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:29:33 2022] GET /identity/v3/projects/1c2139beef6b44b9a0574047298c1af9 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:33.348764 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a751a-23cc-4ffc-a753-e5180605bc35 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:33.422348 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f10d9-d2a9-4d5f-bd9b-ae9d61d25e9a None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:33.442083 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:33.458649 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a751a-23cc-4ffc-a753-e5180605bc35 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:33.462062 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a45411-53cb-481c-b4dc-c09d74c86118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:33.462809 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a45411-53cb-481c-b4dc-c09d74c86118 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:33.463223 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a45411-53cb-481c-b4dc-c09d74c86118 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:33.463637 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a45411-53cb-481c-b4dc-c09d74c86118 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:33.496754 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbff7fcd-68cc-47c8-b30a-e9756433ec2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:33.497689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff7fcd-68cc-47c8-b30a-e9756433ec2e None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:33.498132 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff7fcd-68cc-47c8-b30a-e9756433ec2e None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:33.498546 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff7fcd-68cc-47c8-b30a-e9756433ec2e None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:33.500841 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbff7fcd-68cc-47c8-b30a-e9756433ec2e None admin] RBAC: Adding query filter params (name=member) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:33.501354 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbff7fcd-68cc-47c8-b30a-e9756433ec2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:33.503023 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbff7fcd-68cc-47c8-b30a-e9756433ec2e None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:33.517378 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 216/429] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 30 14:29:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:33.518652 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbff7fcd-68cc-47c8-b30a-e9756433ec2e None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:33.540364 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a45411-53cb-481c-b4dc-c09d74c86118 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:33.541981 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a45411-53cb-481c-b4dc-c09d74c86118 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:33.585042 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:33.643962 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbff7fcd-68cc-47c8-b30a-e9756433ec2e None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:33.648751 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 214/430] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:29:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:29:33.649595 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a45411-53cb-481c-b4dc-c09d74c86118 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:33.672731 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:33.687653 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa9dbdb-cc43-46eb-ba4b-e505a9dff809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:33.688573 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa9dbdb-cc43-46eb-ba4b-e505a9dff809 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:33.689299 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa9dbdb-cc43-46eb-ba4b-e505a9dff809 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:33.690620 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa9dbdb-cc43-46eb-ba4b-e505a9dff809 None admin] PATH_INFO: `/v3/groups/88ad07bcdacd4ef79be074cf74d7f8b2` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:33.691971 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa9dbdb-cc43-46eb-ba4b-e505a9dff809 None admin] RBAC: Authorizing `identity:get_group(group_id=88ad07bcdacd4ef79be074cf74d7f8b2)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:33.693630 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa9dbdb-cc43-46eb-ba4b-e505a9dff809 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:33.701290 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 217/431] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Aug 30 14:29:33 2022] GET /identity/v3/groups/88ad07bcdacd4ef79be074cf74d7f8b2 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:33.701290 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3887edb-da9f-4bf3-948e-0a12ea611b78 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:33.701290 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3887edb-da9f-4bf3-948e-0a12ea611b78 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:33.701552 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3887edb-da9f-4bf3-948e-0a12ea611b78 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:33.703525 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 218/432] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:33.703525 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3887edb-da9f-4bf3-948e-0a12ea611b78 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:33.764156 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a45411-53cb-481c-b4dc-c09d74c86118 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:33.801459 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab3985b-b348-4fce-bf02-a6367ff016e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:33.802074 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab3985b-b348-4fce-bf02-a6367ff016e2 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:33.802290 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab3985b-b348-4fce-bf02-a6367ff016e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:33.802521 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab3985b-b348-4fce-bf02-a6367ff016e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:33.819329 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3887edb-da9f-4bf3-948e-0a12ea611b78 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:33.847056 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:33.863566 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e067f4b-e51c-451b-9c05-83d3c29b217e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:33.864536 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e067f4b-e51c-451b-9c05-83d3c29b217e None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:33.864823 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e067f4b-e51c-451b-9c05-83d3c29b217e None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:33.865152 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e067f4b-e51c-451b-9c05-83d3c29b217e None admin] PATH_INFO: `/v3/projects/1c2139beef6b44b9a0574047298c1af9` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:33.867145 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e067f4b-e51c-451b-9c05-83d3c29b217e None admin] RBAC: Authorizing `identity:get_project(project_id=1c2139beef6b44b9a0574047298c1af9)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:33.868754 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e067f4b-e51c-451b-9c05-83d3c29b217e None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:33.871718 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 219/433] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:29:33 2022] GET /identity/v3/projects/1c2139beef6b44b9a0574047298c1af9 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:33.872706 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e067f4b-e51c-451b-9c05-83d3c29b217e None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:33.877493 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab3985b-b348-4fce-bf02-a6367ff016e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:33.879013 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab3985b-b348-4fce-bf02-a6367ff016e2 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:33.899627 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:33.987578 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e067f4b-e51c-451b-9c05-83d3c29b217e None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:33.999392 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 215/434] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:29:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:29:33.999392 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1779dccc-4899-44c9-b147-ed697ecacf16 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:33.999392 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1779dccc-4899-44c9-b147-ed697ecacf16 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:33.999392 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1779dccc-4899-44c9-b147-ed697ecacf16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:34.015721 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-1779dccc-4899-44c9-b147-ed697ecacf16 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:34.016907 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-1779dccc-4899-44c9-b147-ed697ecacf16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:34.032871 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c1b76e-f426-4c5d-b434-00140d5883d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:34.033482 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c1b76e-f426-4c5d-b434-00140d5883d9 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:34.033749 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c1b76e-f426-4c5d-b434-00140d5883d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:34.034020 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c1b76e-f426-4c5d-b434-00140d5883d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:34.034738 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c1b76e-f426-4c5d-b434-00140d5883d9 None admin] RBAC: Adding query filter params (group.id=88ad07bcdacd4ef79be074cf74d7f8b2, role.id=6a985d5659a3416cba33ca9d30b39f85, scope.project.id=1c2139beef6b44b9a0574047298c1af9) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:34.035041 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c1b76e-f426-4c5d-b434-00140d5883d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:34.036532 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c1b76e-f426-4c5d-b434-00140d5883d9 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:34.044297 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 220/435] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Aug 30 14:29:33 2022] GET /identity/v3/role_assignments?group.id=88ad07bcdacd4ef79be074cf74d7f8b2&scope.project.id=1c2139beef6b44b9a0574047298c1af9&role.id=6a985d5659a3416cba33ca9d30b39f85 => generated 611 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:34.045462 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c1b76e-f426-4c5d-b434-00140d5883d9 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:34.094627 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 216/436] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:34.095751 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1779dccc-4899-44c9-b147-ed697ecacf16 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:34.138641 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:34.153932 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:34.193943 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c1b76e-f426-4c5d-b434-00140d5883d9 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:34.210468 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1779dccc-4899-44c9-b147-ed697ecacf16 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:34.238957 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568a27e-26b1-4b23-a934-11fbf44dbeb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:34.239640 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568a27e-26b1-4b23-a934-11fbf44dbeb2 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:34.239919 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568a27e-26b1-4b23-a934-11fbf44dbeb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:34.240190 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568a27e-26b1-4b23-a934-11fbf44dbeb2 None admin] PATH_INFO: `/v3/groups/88ad07bcdacd4ef79be074cf74d7f8b2` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:34.241327 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8568a27e-26b1-4b23-a934-11fbf44dbeb2 None admin] RBAC: Authorizing `identity:get_group(group_id=88ad07bcdacd4ef79be074cf74d7f8b2)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:34.242751 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8568a27e-26b1-4b23-a934-11fbf44dbeb2 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:34.245325 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 221/437] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Aug 30 14:29:34 2022] GET /identity/v3/groups/88ad07bcdacd4ef79be074cf74d7f8b2 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:34.246465 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51a0a2-9813-474a-9d12-9349512c88ad None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:34.246741 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51a0a2-9813-474a-9d12-9349512c88ad None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:34.247008 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51a0a2-9813-474a-9d12-9349512c88ad None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:34.250124 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 222/438] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:34.254869 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a51a0a2-9813-474a-9d12-9349512c88ad None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:34.260115 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5954272b-2275-4a36-aeeb-3ac9a1f7e94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:34.262204 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954272b-2275-4a36-aeeb-3ac9a1f7e94a None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:34.262552 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954272b-2275-4a36-aeeb-3ac9a1f7e94a None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:34.262932 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954272b-2275-4a36-aeeb-3ac9a1f7e94a None admin] PATH_INFO: `/v3/projects/961f8057321f482d9a5d9202c7bc3c4c` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:34.264218 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5954272b-2275-4a36-aeeb-3ac9a1f7e94a None admin] RBAC: Authorizing `identity:get_project(project_id=961f8057321f482d9a5d9202c7bc3c4c)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:34.265658 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5954272b-2275-4a36-aeeb-3ac9a1f7e94a None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:34.267402 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 217/439] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:29:34 2022] GET /identity/v3/projects/961f8057321f482d9a5d9202c7bc3c4c => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:34.268542 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb855aab-9dd2-494c-bf46-9e83afe1a8cf None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:34.268823 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb855aab-9dd2-494c-bf46-9e83afe1a8cf None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:34.269093 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb855aab-9dd2-494c-bf46-9e83afe1a8cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:34.293031 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-cb855aab-9dd2-494c-bf46-9e83afe1a8cf None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:34.294474 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-cb855aab-9dd2-494c-bf46-9e83afe1a8cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:34.376868 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 218/440] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:34.377912 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb855aab-9dd2-494c-bf46-9e83afe1a8cf None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:34.378330 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a51a0a2-9813-474a-9d12-9349512c88ad None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:34.406021 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:34.418217 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3095988e-94e5-43ec-8199-22084981c74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:34.421909 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3095988e-94e5-43ec-8199-22084981c74d None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:34.421909 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3095988e-94e5-43ec-8199-22084981c74d None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:34.421909 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3095988e-94e5-43ec-8199-22084981c74d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:34.421909 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3095988e-94e5-43ec-8199-22084981c74d None admin] RBAC: Adding query filter params (group.id=88ad07bcdacd4ef79be074cf74d7f8b2, role.id=1c4886a7a7fd402d8221a0158ddcb167, scope.project.id=1c2139beef6b44b9a0574047298c1af9) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:34.421909 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3095988e-94e5-43ec-8199-22084981c74d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:34.423718 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3095988e-94e5-43ec-8199-22084981c74d None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:34.429866 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 223/441] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Aug 30 14:29:34 2022] GET /identity/v3/role_assignments?group.id=88ad07bcdacd4ef79be074cf74d7f8b2&scope.project.id=1c2139beef6b44b9a0574047298c1af9&role.id=1c4886a7a7fd402d8221a0158ddcb167 => generated 611 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:34.435236 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3095988e-94e5-43ec-8199-22084981c74d None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:34.445850 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:34.520872 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb855aab-9dd2-494c-bf46-9e83afe1a8cf None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:34.559792 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3095988e-94e5-43ec-8199-22084981c74d None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:34.575314 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1301cf-80f1-4fca-bd29-78486067eb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:34.576500 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1301cf-80f1-4fca-bd29-78486067eb04 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:34.576791 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1301cf-80f1-4fca-bd29-78486067eb04 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:34.577069 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1301cf-80f1-4fca-bd29-78486067eb04 None admin] PATH_INFO: `/v3/groups/88ad07bcdacd4ef79be074cf74d7f8b2` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:34.579501 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1301cf-80f1-4fca-bd29-78486067eb04 None admin] RBAC: Authorizing `identity:get_group(group_id=88ad07bcdacd4ef79be074cf74d7f8b2)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:34.581200 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1301cf-80f1-4fca-bd29-78486067eb04 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:34.584332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 219/442] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Aug 30 14:29:34 2022] GET /identity/v3/groups/88ad07bcdacd4ef79be074cf74d7f8b2 => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:34.593143 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1301cf-80f1-4fca-bd29-78486067eb04 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:34.605330 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb01146-ee0c-4ff7-8537-24e128cf7927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:34.605973 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb01146-ee0c-4ff7-8537-24e128cf7927 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:34.606252 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb01146-ee0c-4ff7-8537-24e128cf7927 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:34.606524 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb01146-ee0c-4ff7-8537-24e128cf7927 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:34.622280 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb01146-ee0c-4ff7-8537-24e128cf7927 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:34.628476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-dbb01146-ee0c-4ff7-8537-24e128cf7927 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:29:34.628476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:34.628476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:34.628476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:34.628476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:34.628476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:34.628476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:34.628476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:34.628476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:34.628476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:34.628476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:34.628476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:34.628476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:34.628476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:34.628476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:34.628476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:34.629428 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:34.629428 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:34.629428 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:34.629428 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:34.629428 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:34.629428 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:34.629428 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:34.629428 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:34.629428 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:34.629428 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:34.629428 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:34.629428 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:34.629428 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:34.629428 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:34.629428 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:34.629428 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:34.630261 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:34.630261 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:34.630261 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:34.630261 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:34.630261 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:34.630261 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:34.630261 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:34.630261 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:34.630261 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:34.630261 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:34.630261 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:29:34.630261 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:34.630261 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 224/443] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 30 14:29:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:34.635694 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb01146-ee0c-4ff7-8537-24e128cf7927 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:34.698256 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:34.742398 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1301cf-80f1-4fca-bd29-78486067eb04 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:34.758283 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:34.760275 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb01146-ee0c-4ff7-8537-24e128cf7927 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:34.787291 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4525ac87-c39c-4e71-b7c7-5699ab6729de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:34.787923 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525ac87-c39c-4e71-b7c7-5699ab6729de None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:34.788183 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525ac87-c39c-4e71-b7c7-5699ab6729de None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:34.788454 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525ac87-c39c-4e71-b7c7-5699ab6729de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:34.789186 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4525ac87-c39c-4e71-b7c7-5699ab6729de None admin] RBAC: Adding query filter params (group.id=88ad07bcdacd4ef79be074cf74d7f8b2, role.id=6a985d5659a3416cba33ca9d30b39f85, scope.project.id=961f8057321f482d9a5d9202c7bc3c4c) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:34.789492 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4525ac87-c39c-4e71-b7c7-5699ab6729de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:34.790915 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4525ac87-c39c-4e71-b7c7-5699ab6729de None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:34.799567 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 220/444] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Aug 30 14:29:34 2022] GET /identity/v3/role_assignments?group.id=88ad07bcdacd4ef79be074cf74d7f8b2&scope.project.id=961f8057321f482d9a5d9202c7bc3c4c&role.id=6a985d5659a3416cba33ca9d30b39f85 => generated 611 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:34.801761 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094a02e1-1615-44df-8795-d4b06c9595d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:34.801761 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a02e1-1615-44df-8795-d4b06c9595d4 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:34.801976 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a02e1-1615-44df-8795-d4b06c9595d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:34.802250 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a02e1-1615-44df-8795-d4b06c9595d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:34.802935 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094a02e1-1615-44df-8795-d4b06c9595d4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:34.803268 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094a02e1-1615-44df-8795-d4b06c9595d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:34.803667 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4525ac87-c39c-4e71-b7c7-5699ab6729de None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:34.804707 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094a02e1-1615-44df-8795-d4b06c9595d4 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:34.815156 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 225/445] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:29:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:34.820811 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094a02e1-1615-44df-8795-d4b06c9595d4 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:34.953854 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4525ac87-c39c-4e71-b7c7-5699ab6729de None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:34.960804 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094a02e1-1615-44df-8795-d4b06c9595d4 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:34.973367 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:34.993373 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8b0da-fa9b-4c32-9645-27dc043ce7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:34.994017 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8b0da-fa9b-4c32-9645-27dc043ce7fa None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:34.994293 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8b0da-fa9b-4c32-9645-27dc043ce7fa None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:34.994562 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8b0da-fa9b-4c32-9645-27dc043ce7fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:35.005959 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848bcaa9-fde7-49b9-be97-94c4757e18d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:35.006431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b8b0da-fa9b-4c32-9645-27dc043ce7fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:35.006509 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848bcaa9-fde7-49b9-be97-94c4757e18d1 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:35.006753 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848bcaa9-fde7-49b9-be97-94c4757e18d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:35.007026 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848bcaa9-fde7-49b9-be97-94c4757e18d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:35.012061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-28b8b0da-fa9b-4c32-9645-27dc043ce7fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:29:35.012061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:35.012061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:35.012061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:35.012061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:35.012061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:35.012061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:35.012061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:35.012061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:35.012061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:35.012061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:35.012061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:35.012061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:35.012061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:35.012061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:35.012061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:35.013048 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:35.013048 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:35.013048 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:35.013048 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:35.013048 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:35.013048 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:35.013048 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:35.013048 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:35.013048 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:35.013048 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:35.013048 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:35.013048 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:35.013048 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:35.013048 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:35.013048 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:35.013048 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:35.013833 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:35.013833 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:35.013833 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:35.013833 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:35.013833 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:35.013833 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:35.013833 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:35.013833 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:35.013833 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:35.013833 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:35.013833 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:29:35.013833 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:35.013833 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 221/446] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 30 14:29:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:35.020161 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8b0da-fa9b-4c32-9645-27dc043ce7fa None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:35.065741 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:35.094382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848bcaa9-fde7-49b9-be97-94c4757e18d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:35.096001 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848bcaa9-fde7-49b9-be97-94c4757e18d1 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:35.139555 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8b0da-fa9b-4c32-9645-27dc043ce7fa None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:35.180098 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630506a2-9ba0-45f5-b2c6-e5572c917d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:35.180708 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630506a2-9ba0-45f5-b2c6-e5572c917d3a None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:35.180987 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630506a2-9ba0-45f5-b2c6-e5572c917d3a None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:35.181254 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630506a2-9ba0-45f5-b2c6-e5572c917d3a None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:35.181906 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630506a2-9ba0-45f5-b2c6-e5572c917d3a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:35.182200 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630506a2-9ba0-45f5-b2c6-e5572c917d3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:35.183613 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630506a2-9ba0-45f5-b2c6-e5572c917d3a None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:35.192126 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 222/447] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:29:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:35.197529 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630506a2-9ba0-45f5-b2c6-e5572c917d3a None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:35.210965 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 226/448] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:29:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:29:35.211787 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848bcaa9-fde7-49b9-be97-94c4757e18d1 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:35.225070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:35.294654 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:35.298154 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630506a2-9ba0-45f5-b2c6-e5572c917d3a None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:35.313138 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848bcaa9-fde7-49b9-be97-94c4757e18d1 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:35.332978 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6797cee5-9223-4cce-9b04-c52b2a85ba0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:35.333583 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6797cee5-9223-4cce-9b04-c52b2a85ba0b None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:35.333858 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6797cee5-9223-4cce-9b04-c52b2a85ba0b None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:35.334129 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6797cee5-9223-4cce-9b04-c52b2a85ba0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:35.346521 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbbabc3-2d30-4d7f-96c3-d080013da7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:35.347137 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbabc3-2d30-4d7f-96c3-d080013da7cb None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:35.347449 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbabc3-2d30-4d7f-96c3-d080013da7cb None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:35.347722 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbabc3-2d30-4d7f-96c3-d080013da7cb None admin] PATH_INFO: `/v3/projects/961f8057321f482d9a5d9202c7bc3c4c` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:35.348837 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbbabc3-2d30-4d7f-96c3-d080013da7cb None admin] RBAC: Authorizing `identity:get_project(project_id=961f8057321f482d9a5d9202c7bc3c4c)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:35.350332 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbbabc3-2d30-4d7f-96c3-d080013da7cb None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:35.351981 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 227/449] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:29:35 2022] GET /identity/v3/projects/961f8057321f482d9a5d9202c7bc3c4c => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:35.357716 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbbabc3-2d30-4d7f-96c3-d080013da7cb None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:35.403965 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6797cee5-9223-4cce-9b04-c52b2a85ba0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:35.405413 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6797cee5-9223-4cce-9b04-c52b2a85ba0b None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:35.434316 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:35.457104 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbbabc3-2d30-4d7f-96c3-d080013da7cb None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:35.490035 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf5ed9-0b99-4235-a885-4766681f7dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:35.490593 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf5ed9-0b99-4235-a885-4766681f7dbd None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:35.490875 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf5ed9-0b99-4235-a885-4766681f7dbd None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:35.491147 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf5ed9-0b99-4235-a885-4766681f7dbd None admin] PATH_INFO: `/v3/groups/88ad07bcdacd4ef79be074cf74d7f8b2` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:35.492325 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bf5ed9-0b99-4235-a885-4766681f7dbd None admin] RBAC: Authorizing `identity:get_group(group_id=88ad07bcdacd4ef79be074cf74d7f8b2)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:35.493761 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bf5ed9-0b99-4235-a885-4766681f7dbd None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:35.495349 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 228/450] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Aug 30 14:29:35 2022] GET /identity/v3/groups/88ad07bcdacd4ef79be074cf74d7f8b2 => generated 226 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:29:35.501758 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf5ed9-0b99-4235-a885-4766681f7dbd None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:35.503020 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:35.507342 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 223/451] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:29:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 14:29:35.513339 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6797cee5-9223-4cce-9b04-c52b2a85ba0b None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:35.600423 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf5ed9-0b99-4235-a885-4766681f7dbd None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:35.612678 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6797cee5-9223-4cce-9b04-c52b2a85ba0b None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:35.633942 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b2f164-a89f-415a-86a3-9ceacd845590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:35.634532 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b2f164-a89f-415a-86a3-9ceacd845590 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:35.634809 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b2f164-a89f-415a-86a3-9ceacd845590 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:35.635079 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b2f164-a89f-415a-86a3-9ceacd845590 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:35.635832 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b2f164-a89f-415a-86a3-9ceacd845590 None admin] RBAC: Adding query filter params (group.id=88ad07bcdacd4ef79be074cf74d7f8b2, role.id=1c4886a7a7fd402d8221a0158ddcb167, scope.project.id=961f8057321f482d9a5d9202c7bc3c4c) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:35.636136 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b2f164-a89f-415a-86a3-9ceacd845590 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:35.638258 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b2f164-a89f-415a-86a3-9ceacd845590 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:35.643943 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 229/452] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Aug 30 14:29:35 2022] GET /identity/v3/role_assignments?group.id=88ad07bcdacd4ef79be074cf74d7f8b2&scope.project.id=961f8057321f482d9a5d9202c7bc3c4c&role.id=1c4886a7a7fd402d8221a0158ddcb167 => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:29:35.647967 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c69550-c781-43e1-b559-685851fbcd9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:35.648614 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c69550-c781-43e1-b559-685851fbcd9a None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:35.648894 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c69550-c781-43e1-b559-685851fbcd9a None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:35.649171 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c69550-c781-43e1-b559-685851fbcd9a None admin] PATH_INFO: `/v3/projects/2ac908dcbf994029b1c39e157d0de0cb` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:35.650354 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c69550-c781-43e1-b559-685851fbcd9a None admin] RBAC: Authorizing `identity:get_project(project_id=2ac908dcbf994029b1c39e157d0de0cb)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:35.651882 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c69550-c781-43e1-b559-685851fbcd9a None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:35.653598 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 224/453] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:29:35 2022] GET /identity/v3/projects/2ac908dcbf994029b1c39e157d0de0cb => generated 344 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:29:35.659671 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b2f164-a89f-415a-86a3-9ceacd845590 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:35.673593 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:35.765942 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b2f164-a89f-415a-86a3-9ceacd845590 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:35.802352 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3c6f36-3b95-48de-9f12-c0eace015fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:35.802984 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c6f36-3b95-48de-9f12-c0eace015fe5 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:35.803292 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c6f36-3b95-48de-9f12-c0eace015fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:35.803567 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c6f36-3b95-48de-9f12-c0eace015fe5 None admin] PATH_INFO: `/v3/groups/9625519eff7243eab5965376fb01cd09` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:35.804688 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3c6f36-3b95-48de-9f12-c0eace015fe5 None admin] RBAC: Authorizing `identity:get_group(group_id=9625519eff7243eab5965376fb01cd09)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:35.806153 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3c6f36-3b95-48de-9f12-c0eace015fe5 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:35.807750 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 230/454] 10.222.0.30 () {56 vars in 1180 bytes} [Tue Aug 30 14:29:35 2022] GET /identity/v3/groups/9625519eff7243eab5965376fb01cd09 => generated 229 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:35.814399 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c69550-c781-43e1-b559-685851fbcd9a None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:35.881144 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:35.915554 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c69550-c781-43e1-b559-685851fbcd9a None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:35.952802 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613f4079-4afd-4625-a175-2ed4f7663904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:35.953413 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f4079-4afd-4625-a175-2ed4f7663904 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:35.953683 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f4079-4afd-4625-a175-2ed4f7663904 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:35.953948 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f4079-4afd-4625-a175-2ed4f7663904 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:35.954668 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613f4079-4afd-4625-a175-2ed4f7663904 None admin] RBAC: Adding query filter params (group.id=9625519eff7243eab5965376fb01cd09, role.id=3cda0615d87d4af0860a2a5e9b4e2c05, scope.project.id=2ac908dcbf994029b1c39e157d0de0cb) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:35.954970 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613f4079-4afd-4625-a175-2ed4f7663904 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:35.956445 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613f4079-4afd-4625-a175-2ed4f7663904 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:35.962195 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 225/455] 10.222.0.30 () {56 vars in 1376 bytes} [Tue Aug 30 14:29:35 2022] GET /identity/v3/role_assignments?group.id=9625519eff7243eab5965376fb01cd09&scope.project.id=2ac908dcbf994029b1c39e157d0de0cb&role.id=3cda0615d87d4af0860a2a5e9b4e2c05 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:38.210390 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f5ed14-d74b-4330-8935-59d908682a16 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:38.210820 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f5ed14-d74b-4330-8935-59d908682a16 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:38.210938 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f5ed14-d74b-4330-8935-59d908682a16 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:38.212593 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 231/456] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:38.218606 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c04ea-e998-4d31-b775-9019196ee176 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:38.219503 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c04ea-e998-4d31-b775-9019196ee176 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:38.219982 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c04ea-e998-4d31-b775-9019196ee176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:38.239514 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-1b6c04ea-e998-4d31-b775-9019196ee176 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:38.241026 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-1b6c04ea-e998-4d31-b775-9019196ee176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:38.323326 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 226/457] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:38.445151 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc941ff3-71d9-4192-95f5-75961da5ec2b None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:38.445782 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc941ff3-71d9-4192-95f5-75961da5ec2b None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:38.446256 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc941ff3-71d9-4192-95f5-75961da5ec2b None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:38.447851 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 232/458] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:38.453148 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be6b19-64f4-43cd-a0f7-02151b205db2 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:38.453672 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be6b19-64f4-43cd-a0f7-02151b205db2 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:38.454178 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be6b19-64f4-43cd-a0f7-02151b205db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:38.477553 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-85be6b19-64f4-43cd-a0f7-02151b205db2 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:38.479108 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-85be6b19-64f4-43cd-a0f7-02151b205db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:38.551557 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:38.565937 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 227/459] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:38.691438 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc941ff3-71d9-4192-95f5-75961da5ec2b None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:38.796849 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:38.826372 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc941ff3-71d9-4192-95f5-75961da5ec2b None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:38.878697 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e494dcb-f1d9-4c03-994c-08394751f328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:38.879744 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e494dcb-f1d9-4c03-994c-08394751f328 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:38.880220 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e494dcb-f1d9-4c03-994c-08394751f328 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:38.880710 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e494dcb-f1d9-4c03-994c-08394751f328 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:38.901191 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e494dcb-f1d9-4c03-994c-08394751f328 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:38.903623 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e494dcb-f1d9-4c03-994c-08394751f328 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:38.906718 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85be6b19-64f4-43cd-a0f7-02151b205db2 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:38.923410 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-0e494dcb-f1d9-4c03-994c-08394751f328 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:38.923410 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:38.923410 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:38.923410 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:38.923410 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:38.923410 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:38.923410 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:38.923410 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:38.923410 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:38.923410 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:38.923410 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:38.923410 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:38.923410 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:29:38.923410 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:29:38.923410 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:29:38.923410 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:29:38.924353 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:38.924353 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:38.924353 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:38.924353 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:38.924353 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:38.924353 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:38.924353 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:38.924353 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:38.924353 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:38.924353 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:38.924353 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:38.924353 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:38.924353 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:38.924353 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:38.924353 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:38.924353 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:38.924353 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:29:38.925213 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:29:38.925213 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:38.925213 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:38.928788 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 233/460] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:29:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:38.936037 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e494dcb-f1d9-4c03-994c-08394751f328 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:39.027871 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85be6b19-64f4-43cd-a0f7-02151b205db2 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:39.065942 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e494dcb-f1d9-4c03-994c-08394751f328 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:39.067769 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7964fc99-2602-4363-812e-c76b7e8000e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:39.068675 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7964fc99-2602-4363-812e-c76b7e8000e3 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:39.069100 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7964fc99-2602-4363-812e-c76b7e8000e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:39.069539 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7964fc99-2602-4363-812e-c76b7e8000e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:39.080822 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7964fc99-2602-4363-812e-c76b7e8000e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:39.082489 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7964fc99-2602-4363-812e-c76b7e8000e3 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:39.096835 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:39.100021 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-7964fc99-2602-4363-812e-c76b7e8000e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:39.100021 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:39.100021 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:39.100021 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:39.100021 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:39.100021 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:39.100021 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:39.100021 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:39.100021 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:39.100021 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:39.100021 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:39.100021 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:39.100021 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:29:39.100021 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:29:39.100021 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:29:39.100021 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:29:39.101111 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:39.101111 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:39.101111 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:39.101111 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:39.101111 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:39.101111 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:39.101111 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:39.101111 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:39.101111 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:39.101111 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:39.101111 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:39.101111 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:39.101111 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:39.101111 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:39.101111 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:39.101111 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:39.101111 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:29:39.102168 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:29:39.102168 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:39.102168 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:39.104549 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 228/461] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:29:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:39.106060 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219b232d-8a21-4c3c-88b4-e85f892c28de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:39.106548 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d743d2-3afa-4c4b-8453-29e8e2bef10d None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:39.106836 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d743d2-3afa-4c4b-8453-29e8e2bef10d None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:39.107110 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d743d2-3afa-4c4b-8453-29e8e2bef10d None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:39.107339 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b232d-8a21-4c3c-88b4-e85f892c28de None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:39.107756 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b232d-8a21-4c3c-88b4-e85f892c28de None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:39.108164 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b232d-8a21-4c3c-88b4-e85f892c28de None admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:39.108533 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 229/462] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:39.109325 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219b232d-8a21-4c3c-88b4-e85f892c28de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:39.109770 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219b232d-8a21-4c3c-88b4-e85f892c28de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:39.111415 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219b232d-8a21-4c3c-88b4-e85f892c28de None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:39.113136 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d743d2-3afa-4c4b-8453-29e8e2bef10d None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:39.127492 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 234/463] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:29:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:29:39.128971 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1152ad56-b183-4bad-896f-9bd39542ca1a None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:39.129402 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1152ad56-b183-4bad-896f-9bd39542ca1a None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:39.129873 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1152ad56-b183-4bad-896f-9bd39542ca1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:39.150693 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-1152ad56-b183-4bad-896f-9bd39542ca1a None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:39.152919 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-1152ad56-b183-4bad-896f-9bd39542ca1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:39.163866 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:39.225823 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d743d2-3afa-4c4b-8453-29e8e2bef10d None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:39.232007 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 235/464] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:39.233385 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1152ad56-b183-4bad-896f-9bd39542ca1a None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:39.275917 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d4b018-dd0b-43ae-a585-fb2038ea6a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:39.277327 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4b018-dd0b-43ae-a585-fb2038ea6a95 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:39.277762 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4b018-dd0b-43ae-a585-fb2038ea6a95 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:39.278187 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4b018-dd0b-43ae-a585-fb2038ea6a95 None admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:39.278970 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d4b018-dd0b-43ae-a585-fb2038ea6a95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:39.279415 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d4b018-dd0b-43ae-a585-fb2038ea6a95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:39.281432 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d4b018-dd0b-43ae-a585-fb2038ea6a95 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:39.298796 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 230/465] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:29:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:39.300882 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1966f87f-06e2-4e09-b6c8-748138e0747c None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:39.301268 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1966f87f-06e2-4e09-b6c8-748138e0747c None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:39.301659 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1966f87f-06e2-4e09-b6c8-748138e0747c None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:39.304473 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 231/466] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:39.305279 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1966f87f-06e2-4e09-b6c8-748138e0747c None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:39.348545 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1152ad56-b183-4bad-896f-9bd39542ca1a None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:39.371194 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:39.394079 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d97ff33-197c-412c-a0d5-5fd82e8b9ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:39.394687 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d97ff33-197c-412c-a0d5-5fd82e8b9ace None admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:39.394968 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d97ff33-197c-412c-a0d5-5fd82e8b9ace None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:39.395266 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d97ff33-197c-412c-a0d5-5fd82e8b9ace None admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:39.395919 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d97ff33-197c-412c-a0d5-5fd82e8b9ace None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:39.397368 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d97ff33-197c-412c-a0d5-5fd82e8b9ace None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:39.423257 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 236/467] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 30 14:29:39 2022] POST /identity/v3/users => generated 246 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:29:39.424624 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673d32e-d6d1-4f47-91ae-f4b3d22a58cc None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:39.424928 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673d32e-d6d1-4f47-91ae-f4b3d22a58cc None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:39.425176 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673d32e-d6d1-4f47-91ae-f4b3d22a58cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:39.448722 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-9673d32e-d6d1-4f47-91ae-f4b3d22a58cc None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:39.450467 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1966f87f-06e2-4e09-b6c8-748138e0747c None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:39.450722 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-9673d32e-d6d1-4f47-91ae-f4b3d22a58cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:39.453811 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:39.505340 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9d399-989a-44a2-b323-1a7ed6b04449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:39.506880 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9d399-989a-44a2-b323-1a7ed6b04449 None admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:39.507701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9d399-989a-44a2-b323-1a7ed6b04449 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:39.508198 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9d399-989a-44a2-b323-1a7ed6b04449 None admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:39.509332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a9d399-989a-44a2-b323-1a7ed6b04449 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:39.511870 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a9d399-989a-44a2-b323-1a7ed6b04449 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:39.533440 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 237/468] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:39.541489 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 232/469] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 30 14:29:39 2022] POST /identity/v3/users => generated 248 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:29:39.667027 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9673d32e-d6d1-4f47-91ae-f4b3d22a58cc None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:39.784350 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9673d32e-d6d1-4f47-91ae-f4b3d22a58cc None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:39.794550 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:39.826754 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04980df9-8703-4461-b29b-f5138c1af5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:39.827427 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04980df9-8703-4461-b29b-f5138c1af5aa None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:39.827710 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04980df9-8703-4461-b29b-f5138c1af5aa None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:39.827981 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04980df9-8703-4461-b29b-f5138c1af5aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:39.838620 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04980df9-8703-4461-b29b-f5138c1af5aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:39.840739 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04980df9-8703-4461-b29b-f5138c1af5aa None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:39.856757 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-04980df9-8703-4461-b29b-f5138c1af5aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:39.856757 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:39.856757 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:39.856757 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:39.856757 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:39.856757 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:39.856757 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:39.856757 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:39.856757 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:39.856757 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:39.856757 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:39.856757 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:39.856757 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:29:39.856757 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:29:39.856757 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:29:39.856757 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:29:39.857673 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:39.857673 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:39.857673 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:39.857673 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:39.857673 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:39.857673 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:39.857673 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:39.857673 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:39.857673 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:39.857673 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:39.857673 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:39.857673 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:39.857673 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:39.857673 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:39.857673 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:39.857673 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:39.857673 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:29:39.858750 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:29:39.858750 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:39.858750 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:39.858750 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 238/470] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:29:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:39.863741 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9d399-989a-44a2-b323-1a7ed6b04449 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:39.867707 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04980df9-8703-4461-b29b-f5138c1af5aa None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:39.989616 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9d399-989a-44a2-b323-1a7ed6b04449 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:39.993688 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04980df9-8703-4461-b29b-f5138c1af5aa None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:40.028564 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9d1099-5722-445c-ae40-f042f119a150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:40.029180 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d1099-5722-445c-ae40-f042f119a150 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:40.029458 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d1099-5722-445c-ae40-f042f119a150 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:40.029725 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d1099-5722-445c-ae40-f042f119a150 None admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:40.030366 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9d1099-5722-445c-ae40-f042f119a150 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:40.030565 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db26cfae-94b7-428b-9cf2-e884cda8ea01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:40.030686 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9d1099-5722-445c-ae40-f042f119a150 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:40.031217 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db26cfae-94b7-428b-9cf2-e884cda8ea01 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:40.031510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db26cfae-94b7-428b-9cf2-e884cda8ea01 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:40.031786 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db26cfae-94b7-428b-9cf2-e884cda8ea01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:40.032040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9d1099-5722-445c-ae40-f042f119a150 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:40.044520 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db26cfae-94b7-428b-9cf2-e884cda8ea01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:40.045985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 233/471] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:29:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:29:40.046095 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db26cfae-94b7-428b-9cf2-e884cda8ea01 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:40.047427 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8e373-0f30-4888-a9d4-ec618b9d25a5 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:40.047717 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8e373-0f30-4888-a9d4-ec618b9d25a5 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:40.048091 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8e373-0f30-4888-a9d4-ec618b9d25a5 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:40.049654 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 234/472] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:40.054430 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b8e373-0f30-4888-a9d4-ec618b9d25a5 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:40.063070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-db26cfae-94b7-428b-9cf2-e884cda8ea01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:40.063070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:40.063070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:40.063070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:40.063070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:40.063070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:40.063070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:40.063070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:40.063070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:40.063070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:40.063070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:40.063070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:40.063070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:29:40.063070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:29:40.063070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:29:40.063070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:29:40.063999 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:40.063999 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:40.063999 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:40.063999 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:40.063999 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:40.063999 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:40.063999 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:40.063999 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:40.063999 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:40.063999 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:40.063999 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:40.063999 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:40.063999 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:40.063999 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:40.063999 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:40.063999 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:40.063999 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:29:40.065246 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:29:40.065246 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:40.065246 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:40.065246 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 239/473] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:29:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:40.066559 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5cc08c-9ee4-4872-93c6-80ac69be0329 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:40.066559 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5cc08c-9ee4-4872-93c6-80ac69be0329 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:40.066559 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5cc08c-9ee4-4872-93c6-80ac69be0329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:40.085273 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:40.087015 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-4e5cc08c-9ee4-4872-93c6-80ac69be0329 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:40.088564 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-4e5cc08c-9ee4-4872-93c6-80ac69be0329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:40.093599 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:40.166802 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 240/474] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:40.167905 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5cc08c-9ee4-4872-93c6-80ac69be0329 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:40.184723 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b8e373-0f30-4888-a9d4-ec618b9d25a5 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:40.245020 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67a884a-1dfe-4348-9a6c-d40a777766a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:40.245648 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67a884a-1dfe-4348-9a6c-d40a777766a1 None admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:40.251295 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67a884a-1dfe-4348-9a6c-d40a777766a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:40.251295 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67a884a-1dfe-4348-9a6c-d40a777766a1 None admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:40.251295 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67a884a-1dfe-4348-9a6c-d40a777766a1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:40.251295 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67a884a-1dfe-4348-9a6c-d40a777766a1 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:40.281532 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5cc08c-9ee4-4872-93c6-80ac69be0329 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:40.285571 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 235/475] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 30 14:29:40 2022] POST /identity/v3/users => generated 248 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:29:40.322188 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5499eb8-de39-44ff-833b-2f91f5ba0600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:40.322816 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5499eb8-de39-44ff-833b-2f91f5ba0600 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:40.323095 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5499eb8-de39-44ff-833b-2f91f5ba0600 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:40.323416 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5499eb8-de39-44ff-833b-2f91f5ba0600 None admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:40.324781 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5499eb8-de39-44ff-833b-2f91f5ba0600 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:40.325334 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5499eb8-de39-44ff-833b-2f91f5ba0600 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:40.326764 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5499eb8-de39-44ff-833b-2f91f5ba0600 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:40.336705 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:40.342209 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 241/476] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:29:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:40.348178 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67a884a-1dfe-4348-9a6c-d40a777766a1 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:40.455973 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:40.474799 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67a884a-1dfe-4348-9a6c-d40a777766a1 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:40.513168 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a990578-ec17-434f-9239-8e80f63c4c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:40.513699 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a990578-ec17-434f-9239-8e80f63c4c89 None admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:40.513972 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a990578-ec17-434f-9239-8e80f63c4c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:40.514242 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a990578-ec17-434f-9239-8e80f63c4c89 None admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:40.514964 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a990578-ec17-434f-9239-8e80f63c4c89 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:40.516443 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a990578-ec17-434f-9239-8e80f63c4c89 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:40.540677 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 236/477] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 30 14:29:40 2022] POST /identity/v3/users => generated 249 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:29:40.582126 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5499eb8-de39-44ff-833b-2f91f5ba0600 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:40.694035 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5499eb8-de39-44ff-833b-2f91f5ba0600 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:40.730417 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cb5f2a-e258-436b-b366-fa952dae2384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:40.731096 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cb5f2a-e258-436b-b366-fa952dae2384 None admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:40.731478 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cb5f2a-e258-436b-b366-fa952dae2384 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:40.731755 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cb5f2a-e258-436b-b366-fa952dae2384 None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:40.732430 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cb5f2a-e258-436b-b366-fa952dae2384 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:40.733798 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cb5f2a-e258-436b-b366-fa952dae2384 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:40.743666 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.sql.core [None req-c0cb5f2a-e258-436b-b366-fa952dae2384 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 30 14:29:40.743666 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 30 14:29:40.743666 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [parameters: {'id': 'db64141c08c8496c982205200bed0d14', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 30 14:29:40.743666 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81689) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 30 14:29:40.760974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-c0cb5f2a-e258-436b-b366-fa952dae2384 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 30 14:29:40.760974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:40.760974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 14:29:40.760974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 14:29:40.760974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 14:29:40.760974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 14:29:40.760974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 14:29:40.760974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 14:29:40.760974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 14:29:40.760974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application conn.query(q) Aug 30 14:29:40.760974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 14:29:40.760974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 14:29:40.760974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 14:29:40.760974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application result.read() Aug 30 14:29:40.760974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 14:29:40.763500 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 14:29:40.763500 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 14:29:40.763500 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 14:29:40.763500 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 14:29:40.763500 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 14:29:40.763500 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 14:29:40.763500 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 14:29:40.763500 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 30 14:29:40.763500 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:40.763500 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 30 14:29:40.763500 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:40.763500 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:40.763500 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 30 14:29:40.763500 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 30 14:29:40.763500 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 30 14:29:40.763500 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return ref.to_dict() Aug 30 14:29:40.763500 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 14:29:40.763500 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application next(self.gen) Aug 30 14:29:40.765701 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 30 14:29:40.765701 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application yield resource Aug 30 14:29:40.765701 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 14:29:40.765701 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application next(self.gen) Aug 30 14:29:40.765701 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 30 14:29:40.765701 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application self.session.rollback() Aug 30 14:29:40.765701 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 30 14:29:40.765701 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application self.force_reraise() Aug 30 14:29:40.765701 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 30 14:29:40.765701 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise self.value Aug 30 14:29:40.765701 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 30 14:29:40.765701 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 30 14:29:40.765701 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 30 14:29:40.765701 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application session.commit() Aug 30 14:29:40.765701 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 30 14:29:40.765701 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 30 14:29:40.765701 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 30 14:29:40.769533 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application self._prepare_impl() Aug 30 14:29:40.769533 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 30 14:29:40.769533 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application self.session.flush() Aug 30 14:29:40.769533 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 30 14:29:40.769533 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application self._flush(objects) Aug 30 14:29:40.769533 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 30 14:29:40.769533 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 30 14:29:40.769533 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 30 14:29:40.769533 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application compat.raise_( Aug 30 14:29:40.769533 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 14:29:40.769533 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception Aug 30 14:29:40.769533 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 30 14:29:40.769533 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application flush_context.execute() Aug 30 14:29:40.769533 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 30 14:29:40.769533 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rec.execute(self) Aug 30 14:29:40.769533 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 30 14:29:40.769533 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 30 14:29:40.770988 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 30 14:29:40.770988 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 30 14:29:40.770988 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 30 14:29:40.770988 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 30 14:29:40.770988 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 30 14:29:40.770988 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 30 14:29:40.770988 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 30 14:29:40.770988 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 30 14:29:40.770988 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 30 14:29:40.770988 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 30 14:29:40.770988 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 30 14:29:40.770988 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 30 14:29:40.770988 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 30 14:29:40.770988 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 30 14:29:40.770988 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 14:29:40.770988 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception Aug 30 14:29:40.772002 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 14:29:40.772002 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 14:29:40.772002 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 14:29:40.772002 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 14:29:40.772002 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 14:29:40.772002 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 14:29:40.772002 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 14:29:40.772002 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application conn.query(q) Aug 30 14:29:40.772002 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 14:29:40.772002 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 14:29:40.772002 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 14:29:40.772002 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application result.read() Aug 30 14:29:40.772002 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 14:29:40.772002 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 14:29:40.772002 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 14:29:40.772002 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 14:29:40.772961 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 14:29:40.772961 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 14:29:40.772961 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 14:29:40.772961 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 14:29:40.772961 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 30 14:29:40.772961 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 30 14:29:40.772961 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [parameters: {'id': 'db64141c08c8496c982205200bed0d14', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 30 14:29:40.772961 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 30 14:29:40.772961 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:40.772961 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:29:40.772961 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:40.772961 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:40.772961 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:40.772961 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:40.772961 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:40.772961 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:40.774004 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:40.774004 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:40.774004 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:40.774004 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:40.774004 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:40.774004 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:40.774004 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 30 14:29:40.774004 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 30 14:29:40.774004 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:40.774004 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:40.774004 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 30 14:29:40.774004 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 30 14:29:40.774004 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 30 14:29:40.774004 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 30 14:29:40.774004 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 30 14:29:40.774004 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:40.775112 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 242/478] 10.222.0.30 () {60 vars in 1131 bytes} [Tue Aug 30 14:29:40 2022] POST /identity/v3/roles => generated 144 bytes in 181 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:40.775189 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a990578-ec17-434f-9239-8e80f63c4c89 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:40.893919 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a990578-ec17-434f-9239-8e80f63c4c89 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:40.924772 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:40.930985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802b90e3-c09d-4f0c-9aa9-c16c58002790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:40.931753 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b90e3-c09d-4f0c-9aa9-c16c58002790 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:40.932280 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b90e3-c09d-4f0c-9aa9-c16c58002790 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:40.932558 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b90e3-c09d-4f0c-9aa9-c16c58002790 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:40.946215 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802b90e3-c09d-4f0c-9aa9-c16c58002790 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:40.952552 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-802b90e3-c09d-4f0c-9aa9-c16c58002790 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:29:40.952552 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:40.952552 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:40.952552 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:40.952552 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:40.952552 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:40.952552 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:40.952552 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:40.952552 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:40.952552 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:40.952552 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:40.952552 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:40.952552 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:40.952552 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:40.952552 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:40.952552 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:40.954696 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:40.954696 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:40.954696 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:40.954696 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:40.954696 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:40.954696 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:40.954696 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:40.954696 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:40.954696 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:40.954696 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:40.954696 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:40.954696 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:40.954696 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:40.954696 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:40.954696 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:40.954696 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:40.956738 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:40.956738 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:40.956738 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:40.956738 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:40.956738 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:40.956738 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:40.956738 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:40.956738 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:40.956738 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:40.956738 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:40.956738 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 14:29:40.956738 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:40.956738 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 237/479] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 30 14:29:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:40.960053 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cb5f2a-e258-436b-b366-fa952dae2384 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:41.029223 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:41.071175 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cb5f2a-e258-436b-b366-fa952dae2384 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:41.108668 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3efc29-ce47-46ad-8e9f-28e0c0062230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:41.109283 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3efc29-ce47-46ad-8e9f-28e0c0062230 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:41.109562 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3efc29-ce47-46ad-8e9f-28e0c0062230 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:41.109834 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3efc29-ce47-46ad-8e9f-28e0c0062230 None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:41.110499 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3efc29-ce47-46ad-8e9f-28e0c0062230 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:41.110805 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3efc29-ce47-46ad-8e9f-28e0c0062230 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:41.112219 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3efc29-ce47-46ad-8e9f-28e0c0062230 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:41.126123 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 243/480] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:29:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:41.998805 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20499e87-e759-4522-8cdc-08c856698f8c None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:41.999426 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20499e87-e759-4522-8cdc-08c856698f8c None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:41.999426 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20499e87-e759-4522-8cdc-08c856698f8c None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:42.001020 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 238/481] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:42.002484 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a0835-b669-4256-80c0-14e250cb5a02 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:42.002946 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a0835-b669-4256-80c0-14e250cb5a02 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:42.003126 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a0835-b669-4256-80c0-14e250cb5a02 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:42.004340 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 244/482] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:42.009325 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b960f95-35e8-4367-a0bc-44452e1e0979 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:42.009600 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b960f95-35e8-4367-a0bc-44452e1e0979 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:42.009871 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b960f95-35e8-4367-a0bc-44452e1e0979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:42.016229 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe88fef9-0d95-4e58-a5e5-3961fe9438ff None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:42.016508 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe88fef9-0d95-4e58-a5e5-3961fe9438ff None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:42.016782 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe88fef9-0d95-4e58-a5e5-3961fe9438ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:42.036167 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-8b960f95-35e8-4367-a0bc-44452e1e0979 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:42.037346 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-8b960f95-35e8-4367-a0bc-44452e1e0979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:42.054376 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-fe88fef9-0d95-4e58-a5e5-3961fe9438ff None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:42.055580 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-fe88fef9-0d95-4e58-a5e5-3961fe9438ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:42.172728 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 239/483] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:42.174882 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 245/484] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:29:42.508073 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b960f95-35e8-4367-a0bc-44452e1e0979 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:42.579281 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe88fef9-0d95-4e58-a5e5-3961fe9438ff None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:42.631693 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b960f95-35e8-4367-a0bc-44452e1e0979 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:42.680343 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:42.693351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:42.702088 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ee3486-2d94-487c-a697-1d6defc156e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:42.702732 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee3486-2d94-487c-a697-1d6defc156e2 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:42.703008 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee3486-2d94-487c-a697-1d6defc156e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:42.703308 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee3486-2d94-487c-a697-1d6defc156e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:42.728244 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ee3486-2d94-487c-a697-1d6defc156e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:42.734135 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-69ee3486-2d94-487c-a697-1d6defc156e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:29:42.734135 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:42.734135 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:42.734135 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:42.734135 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:42.734135 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:42.734135 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:42.734135 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:42.734135 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:42.734135 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:42.734135 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:42.734135 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:42.734135 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:42.734135 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:42.734135 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:42.734135 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:42.736185 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:42.736185 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:42.736185 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:42.736185 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:42.736185 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:42.736185 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:42.736185 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:42.736185 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:42.736185 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:42.736185 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:42.736185 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:42.736185 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:42.736185 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:42.736185 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:42.736185 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:42.736185 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:42.738072 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:42.738072 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:42.738072 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:42.738072 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:42.738072 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:42.738072 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:42.738072 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:42.738072 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:42.738072 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:42.738072 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:42.738072 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:29:42.738072 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:42.743228 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 240/485] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 30 14:29:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:42.744852 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ee3486-2d94-487c-a697-1d6defc156e2 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:42.751021 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe88fef9-0d95-4e58-a5e5-3961fe9438ff None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:42.804056 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3243eb78-52e9-4a33-8d82-951534c53145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:42.804659 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243eb78-52e9-4a33-8d82-951534c53145 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:42.804944 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243eb78-52e9-4a33-8d82-951534c53145 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:42.805223 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243eb78-52e9-4a33-8d82-951534c53145 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:42.825608 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3243eb78-52e9-4a33-8d82-951534c53145 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:42.834859 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-3243eb78-52e9-4a33-8d82-951534c53145 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:29:42.834859 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:42.834859 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:42.834859 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:42.834859 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:42.834859 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:42.834859 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:42.834859 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:42.834859 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:42.834859 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:42.834859 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:42.834859 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:42.834859 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:42.834859 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:42.834859 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:42.834859 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:42.837007 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:42.837007 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:42.837007 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:42.837007 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:42.837007 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:42.837007 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:42.837007 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:42.837007 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:42.837007 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:42.837007 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:42.837007 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:42.837007 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:42.837007 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:42.837007 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:42.837007 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:42.837007 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:42.838902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:42.838902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:42.838902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:42.838902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:42.838902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:42.838902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:42.838902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:42.838902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:42.838902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:42.838902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:42.838902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:29:42.838902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:42.838902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 246/486] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 30 14:29:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:42.845419 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3243eb78-52e9-4a33-8d82-951534c53145 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:42.880549 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ee3486-2d94-487c-a697-1d6defc156e2 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:42.936614 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c0e0ff-fc94-440c-8c02-d363ffd55d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:42.937222 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c0e0ff-fc94-440c-8c02-d363ffd55d5a None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:42.937608 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c0e0ff-fc94-440c-8c02-d363ffd55d5a None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:42.937926 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c0e0ff-fc94-440c-8c02-d363ffd55d5a None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:42.938711 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c0e0ff-fc94-440c-8c02-d363ffd55d5a None admin] RBAC: Adding query filter params (name=service) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:42.939036 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c0e0ff-fc94-440c-8c02-d363ffd55d5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:42.940455 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c0e0ff-fc94-440c-8c02-d363ffd55d5a None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:42.950498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 241/487] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 30 14:29:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:42.952425 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84193a43-d246-4e01-888d-c4cc7bc726c4 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:42.952711 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84193a43-d246-4e01-888d-c4cc7bc726c4 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:42.952984 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84193a43-d246-4e01-888d-c4cc7bc726c4 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:42.954325 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 242/488] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:42.955244 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a431c-7118-4bca-9585-cc7781df6baa None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:42.955534 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a431c-7118-4bca-9585-cc7781df6baa None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:42.955819 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a431c-7118-4bca-9585-cc7781df6baa None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:42.957137 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 243/489] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:42.957652 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a431c-7118-4bca-9585-cc7781df6baa None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:43.002633 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3243eb78-52e9-4a33-8d82-951534c53145 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:43.008029 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:43.021414 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:43.042757 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edba5401-29fd-47c5-9838-f4745509b914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:43.043506 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edba5401-29fd-47c5-9838-f4745509b914 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:43.043856 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edba5401-29fd-47c5-9838-f4745509b914 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:43.044197 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edba5401-29fd-47c5-9838-f4745509b914 None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:43.044926 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edba5401-29fd-47c5-9838-f4745509b914 None admin] RBAC: Adding query filter params (name=service) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:43.045299 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edba5401-29fd-47c5-9838-f4745509b914 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:43.046684 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edba5401-29fd-47c5-9838-f4745509b914 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:43.061438 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 247/490] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 30 14:29:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:43.061438 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f780cf7-5f90-4a4c-b3be-f7e45651fb9a None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:43.061438 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f780cf7-5f90-4a4c-b3be-f7e45651fb9a None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:43.061438 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f780cf7-5f90-4a4c-b3be-f7e45651fb9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:43.073198 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a431c-7118-4bca-9585-cc7781df6baa None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:43.077421 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-3f780cf7-5f90-4a4c-b3be-f7e45651fb9a None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:43.078616 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-3f780cf7-5f90-4a4c-b3be-f7e45651fb9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:43.110029 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ad6c5-bc02-4ad2-b563-a8fecb58ae5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:43.110660 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ad6c5-bc02-4ad2-b563-a8fecb58ae5a None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:43.110941 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ad6c5-bc02-4ad2-b563-a8fecb58ae5a None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:43.111237 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ad6c5-bc02-4ad2-b563-a8fecb58ae5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:43.122720 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ad6c5-bc02-4ad2-b563-a8fecb58ae5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:43.124219 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ad6c5-bc02-4ad2-b563-a8fecb58ae5a None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:43.139178 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-a77ad6c5-bc02-4ad2-b563-a8fecb58ae5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:43.139178 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:43.139178 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:43.139178 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:43.139178 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:43.139178 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:43.139178 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:43.139178 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:43.139178 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:43.139178 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:43.139178 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:43.139178 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:43.139178 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:29:43.139178 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:29:43.139178 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:29:43.139178 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:29:43.140105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:43.140105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:43.140105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:43.140105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:43.140105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:43.140105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:43.140105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:43.140105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:43.140105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:43.140105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:43.140105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:43.140105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:43.140105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:43.140105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:43.140105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:43.140105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:43.140105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:29:43.140962 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:29:43.140962 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:43.140962 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:43.140962 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 244/491] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:29:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:43.141512 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26212e74-5f0e-40dc-a9f3-1cf9639ee2c9 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:43.141713 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26212e74-5f0e-40dc-a9f3-1cf9639ee2c9 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:43.141986 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26212e74-5f0e-40dc-a9f3-1cf9639ee2c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:43.160320 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-26212e74-5f0e-40dc-a9f3-1cf9639ee2c9 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:43.161527 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-26212e74-5f0e-40dc-a9f3-1cf9639ee2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:43.161956 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 248/492] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:43.163112 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f780cf7-5f90-4a4c-b3be-f7e45651fb9a None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:43.269726 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 245/493] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:43.271590 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26212e74-5f0e-40dc-a9f3-1cf9639ee2c9 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:43.289583 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f780cf7-5f90-4a4c-b3be-f7e45651fb9a None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:43.300152 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:43.311759 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:43.348860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f75a72-dcb4-46c1-9165-5510b2dc7c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:43.349504 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f75a72-dcb4-46c1-9165-5510b2dc7c14 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:43.350896 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f75a72-dcb4-46c1-9165-5510b2dc7c14 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:43.351183 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f75a72-dcb4-46c1-9165-5510b2dc7c14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:43.375563 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f75a72-dcb4-46c1-9165-5510b2dc7c14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:43.379150 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f75a72-dcb4-46c1-9165-5510b2dc7c14 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:43.403510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-c4f75a72-dcb4-46c1-9165-5510b2dc7c14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:43.403510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:43.403510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:43.403510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:43.403510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:43.403510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:43.403510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:43.403510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:43.403510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:43.403510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:43.403510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:43.403510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:43.403510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:29:43.403510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:29:43.403510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:29:43.403510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:29:43.404449 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:43.404449 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:43.404449 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:43.404449 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:43.404449 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:43.404449 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:43.404449 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:43.404449 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:43.404449 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:43.404449 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:43.404449 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:43.404449 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:43.404449 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:43.404449 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:43.404449 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:43.404449 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:43.404449 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:29:43.405285 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:29:43.405285 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:43.405285 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:43.415553 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 249/494] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:29:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:43.416819 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f75a72-dcb4-46c1-9165-5510b2dc7c14 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:43.434933 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26212e74-5f0e-40dc-a9f3-1cf9639ee2c9 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:43.479787 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081b138d-f078-473f-b7a1-f4cb7e9ba2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:43.482882 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b138d-f078-473f-b7a1-f4cb7e9ba2bf None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:43.483171 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b138d-f078-473f-b7a1-f4cb7e9ba2bf None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:43.483470 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b138d-f078-473f-b7a1-f4cb7e9ba2bf None admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:43.484120 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081b138d-f078-473f-b7a1-f4cb7e9ba2bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:43.484425 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081b138d-f078-473f-b7a1-f4cb7e9ba2bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:43.489634 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081b138d-f078-473f-b7a1-f4cb7e9ba2bf None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:43.513771 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 246/495] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:29:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:43.520498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081b138d-f078-473f-b7a1-f4cb7e9ba2bf None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:43.576041 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f75a72-dcb4-46c1-9165-5510b2dc7c14 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:43.615196 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d1858d-3210-430c-b8d0-cbb7a5647d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:43.615833 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1858d-3210-430c-b8d0-cbb7a5647d10 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:43.616109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1858d-3210-430c-b8d0-cbb7a5647d10 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:43.616382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1858d-3210-430c-b8d0-cbb7a5647d10 None admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:43.617024 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d1858d-3210-430c-b8d0-cbb7a5647d10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:43.617324 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d1858d-3210-430c-b8d0-cbb7a5647d10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:43.618708 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d1858d-3210-430c-b8d0-cbb7a5647d10 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:43.626031 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:43.633166 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 250/496] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:29:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:43.636074 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d1858d-3210-430c-b8d0-cbb7a5647d10 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:43.646833 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081b138d-f078-473f-b7a1-f4cb7e9ba2bf None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:43.660350 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:43.684096 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0c27e6-f365-4e36-be6c-99dc20e8b5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:43.684718 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c27e6-f365-4e36-be6c-99dc20e8b5d4 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:43.684997 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c27e6-f365-4e36-be6c-99dc20e8b5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:43.685269 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c27e6-f365-4e36-be6c-99dc20e8b5d4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:43.693618 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0c27e6-f365-4e36-be6c-99dc20e8b5d4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:43.695141 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0c27e6-f365-4e36-be6c-99dc20e8b5d4 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:43.707087 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-dc0c27e6-f365-4e36-be6c-99dc20e8b5d4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:29:43.707087 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:43.707087 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:43.707087 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:43.707087 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:43.707087 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:43.707087 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:43.707087 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:43.707087 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:43.707087 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:43.707087 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:43.707087 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:43.707087 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:29:43.707087 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:29:43.707087 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:29:43.707087 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:29:43.708266 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:43.708266 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:43.708266 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:29:43.708266 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:29:43.708266 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:29:43.708266 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:29:43.708266 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:43.708266 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:43.708266 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:43.708266 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:43.708266 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:43.708266 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:43.708266 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:43.708266 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:43.708266 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:43.708266 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:43.708266 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:43.708266 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:43.709166 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:43.709166 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:43.709166 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:29:43.709166 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:29:43.709166 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:29:43.709166 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:29:43.709166 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:29:43.709166 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:29:43.709166 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:29:43.709166 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:43.709166 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 247/497] 10.222.0.30 () {56 vars in 1093 bytes} [Tue Aug 30 14:29:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:43.709166 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0c27e6-f365-4e36-be6c-99dc20e8b5d4 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:43.744662 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d1858d-3210-430c-b8d0-cbb7a5647d10 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:43.778552 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b427ad-a97e-4938-afff-779b70e05513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:43.779251 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b427ad-a97e-4938-afff-779b70e05513 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:43.779546 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b427ad-a97e-4938-afff-779b70e05513 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:43.779820 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b427ad-a97e-4938-afff-779b70e05513 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:43.789697 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b427ad-a97e-4938-afff-779b70e05513 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:43.794714 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-a7b427ad-a97e-4938-afff-779b70e05513 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:29:43.794714 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:43.794714 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:43.794714 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:43.794714 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:43.794714 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:43.794714 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:43.794714 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:43.794714 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:43.794714 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:43.794714 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:43.794714 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:43.794714 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:43.794714 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:43.794714 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:43.794714 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:43.795672 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:43.795672 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:43.795672 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:43.795672 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:43.795672 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:43.795672 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:43.795672 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:43.795672 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:43.795672 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:43.795672 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:43.795672 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:43.795672 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:43.795672 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:43.795672 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:43.795672 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:43.795672 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:43.796461 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:43.796461 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:43.796461 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:43.796461 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:43.796461 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:43.796461 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:43.796461 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:43.796461 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:43.796461 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:43.796461 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:43.796461 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:29:43.796461 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:43.796461 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 251/498] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 30 14:29:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:43.797123 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b427ad-a97e-4938-afff-779b70e05513 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:43.811959 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0c27e6-f365-4e36-be6c-99dc20e8b5d4 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:43.846082 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43d21db-da92-4dae-a26a-d761d2a31713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:43.846633 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d21db-da92-4dae-a26a-d761d2a31713 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:43.846905 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d21db-da92-4dae-a26a-d761d2a31713 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:43.847170 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d21db-da92-4dae-a26a-d761d2a31713 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:43.857176 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43d21db-da92-4dae-a26a-d761d2a31713 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:43.862380 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-b43d21db-da92-4dae-a26a-d761d2a31713 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:29:43.862380 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:43.862380 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:43.862380 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:43.862380 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:43.862380 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:43.862380 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:43.862380 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:43.862380 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:43.862380 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:43.862380 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:43.862380 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:43.862380 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:43.862380 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:43.862380 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:43.862380 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:43.863330 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:43.863330 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:43.863330 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:43.863330 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:43.863330 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:43.863330 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:43.863330 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:43.863330 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:43.863330 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:43.863330 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:43.863330 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:43.863330 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:43.863330 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:43.863330 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:43.863330 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:43.863330 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:43.864122 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:43.864122 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:43.864122 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:43.864122 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:43.864122 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:43.864122 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:43.864122 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:43.864122 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:43.864122 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:43.864122 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:43.864122 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:29:43.864122 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:43.864122 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 248/499] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 30 14:29:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:43.864778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7befc781-6812-4094-b8e1-41ecaccf277e None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:43.865085 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7befc781-6812-4094-b8e1-41ecaccf277e None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:43.865188 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7befc781-6812-4094-b8e1-41ecaccf277e None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:43.866669 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 249/500] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:43.867071 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7befc781-6812-4094-b8e1-41ecaccf277e None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:43.876508 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:43.882794 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:43.903096 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b427ad-a97e-4938-afff-779b70e05513 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:43.936498 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d9041e-4771-42ad-8457-1042d50982f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:43.937127 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9041e-4771-42ad-8457-1042d50982f0 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:43.937413 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9041e-4771-42ad-8457-1042d50982f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:43.937690 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9041e-4771-42ad-8457-1042d50982f0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:43.945014 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d9041e-4771-42ad-8457-1042d50982f0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:43.946474 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d9041e-4771-42ad-8457-1042d50982f0 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:43.959767 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-05d9041e-4771-42ad-8457-1042d50982f0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 30 14:29:43.959767 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:43.959767 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:43.959767 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:43.959767 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:43.959767 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:43.959767 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:43.959767 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:43.959767 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:43.959767 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:43.959767 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:43.959767 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:43.959767 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:29:43.959767 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:29:43.959767 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:29:43.959767 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:29:43.960736 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:43.960736 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:43.960736 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:29:43.960736 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:29:43.960736 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:29:43.960736 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:29:43.960736 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:43.960736 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:43.960736 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:43.960736 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:43.960736 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:43.960736 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:43.960736 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:43.960736 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:43.960736 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:43.960736 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:43.960736 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:43.960736 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:43.961629 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:43.961629 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:43.961629 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:29:43.961629 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:29:43.961629 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:29:43.961629 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:29:43.961629 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:29:43.961629 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:29:43.961629 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 30 14:29:43.961629 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:43.961629 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 252/501] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:29:43 2022] GET /identity/v3/users/glance => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:43.961629 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d9041e-4771-42ad-8457-1042d50982f0 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:43.976418 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7befc781-6812-4094-b8e1-41ecaccf277e None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:44.012105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d1465e-cf56-4d28-af1f-684c4f139291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:44.012705 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1465e-cf56-4d28-af1f-684c4f139291 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:44.012998 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1465e-cf56-4d28-af1f-684c4f139291 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:44.013267 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1465e-cf56-4d28-af1f-684c4f139291 None admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:44.014025 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d1465e-cf56-4d28-af1f-684c4f139291 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:44.014326 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d1465e-cf56-4d28-af1f-684c4f139291 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:44.015807 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d1465e-cf56-4d28-af1f-684c4f139291 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:44.051812 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 250/502] 10.222.0.30 () {56 vars in 1133 bytes} [Tue Aug 30 14:29:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:44.056462 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d1465e-cf56-4d28-af1f-684c4f139291 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:44.060639 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d9041e-4771-42ad-8457-1042d50982f0 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:44.093382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a4f3b-fa4f-47d9-87ef-d3ae3bbcecfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:44.093980 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a4f3b-fa4f-47d9-87ef-d3ae3bbcecfa None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:44.094263 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a4f3b-fa4f-47d9-87ef-d3ae3bbcecfa None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:44.094532 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a4f3b-fa4f-47d9-87ef-d3ae3bbcecfa None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:44.095182 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578a4f3b-fa4f-47d9-87ef-d3ae3bbcecfa None admin] RBAC: Adding query filter params (name=service) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:44.095516 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578a4f3b-fa4f-47d9-87ef-d3ae3bbcecfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:44.096954 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578a4f3b-fa4f-47d9-87ef-d3ae3bbcecfa None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:44.104657 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 253/503] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 30 14:29:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:44.106610 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe787c0-748d-4a93-bbc0-6fe34abb650b None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:44.106896 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe787c0-748d-4a93-bbc0-6fe34abb650b None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:44.107173 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe787c0-748d-4a93-bbc0-6fe34abb650b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:44.113782 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:44.120139 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:44.126989 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-cfe787c0-748d-4a93-bbc0-6fe34abb650b None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:44.128231 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-cfe787c0-748d-4a93-bbc0-6fe34abb650b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:44.161063 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d1465e-cf56-4d28-af1f-684c4f139291 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:44.194648 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 254/504] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:44.195506 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe787c0-748d-4a93-bbc0-6fe34abb650b None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:44.208388 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2405da8c-1c1f-4b51-a0d7-5a47dd6dbede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:44.209222 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2405da8c-1c1f-4b51-a0d7-5a47dd6dbede None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:44.209506 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2405da8c-1c1f-4b51-a0d7-5a47dd6dbede None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:44.210273 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2405da8c-1c1f-4b51-a0d7-5a47dd6dbede None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:44.211518 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2405da8c-1c1f-4b51-a0d7-5a47dd6dbede None admin] RBAC: Adding query filter params (name=service) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:44.211518 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2405da8c-1c1f-4b51-a0d7-5a47dd6dbede None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:44.212953 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2405da8c-1c1f-4b51-a0d7-5a47dd6dbede None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:44.223592 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 251/505] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 30 14:29:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:44.224626 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2405da8c-1c1f-4b51-a0d7-5a47dd6dbede None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:44.311033 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe787c0-748d-4a93-bbc0-6fe34abb650b None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:44.331819 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2405da8c-1c1f-4b51-a0d7-5a47dd6dbede None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:44.345367 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:44.351848 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca44fe-5d3d-41e8-a90f-c4d672ca67fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:44.352412 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca44fe-5d3d-41e8-a90f-c4d672ca67fe None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:44.352693 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca44fe-5d3d-41e8-a90f-c4d672ca67fe None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:44.352967 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca44fe-5d3d-41e8-a90f-c4d672ca67fe None admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:44.354127 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eca44fe-5d3d-41e8-a90f-c4d672ca67fe None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:44.354584 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eca44fe-5d3d-41e8-a90f-c4d672ca67fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:44.356094 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eca44fe-5d3d-41e8-a90f-c4d672ca67fe None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:44.359076 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:44.368611 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f65edc1-b53e-4735-a3cb-54c11858d207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:44.369236 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f65edc1-b53e-4735-a3cb-54c11858d207 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:44.369488 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f65edc1-b53e-4735-a3cb-54c11858d207 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:44.369756 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f65edc1-b53e-4735-a3cb-54c11858d207 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:44.398227 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 255/506] 10.222.0.30 () {56 vars in 1137 bytes} [Tue Aug 30 14:29:44 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:44.398227 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca44fe-5d3d-41e8-a90f-c4d672ca67fe None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:44.446683 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f65edc1-b53e-4735-a3cb-54c11858d207 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:44.448279 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f65edc1-b53e-4735-a3cb-54c11858d207 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:44.499150 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca44fe-5d3d-41e8-a90f-c4d672ca67fe None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:44.534077 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e596843-25be-438c-8883-0e952ce7cebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:44.534702 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e596843-25be-438c-8883-0e952ce7cebe None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:44.534980 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e596843-25be-438c-8883-0e952ce7cebe None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:44.535281 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e596843-25be-438c-8883-0e952ce7cebe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:44.544996 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e596843-25be-438c-8883-0e952ce7cebe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:44.546442 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e596843-25be-438c-8883-0e952ce7cebe None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:44.548214 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 252/507] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:29:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:29:44.552811 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f65edc1-b53e-4735-a3cb-54c11858d207 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:44.561704 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-5e596843-25be-438c-8883-0e952ce7cebe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:44.561704 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:44.561704 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:44.561704 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:44.561704 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:44.561704 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:44.561704 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:44.561704 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:44.561704 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:44.561704 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:44.561704 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:44.561704 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:44.561704 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:29:44.561704 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:29:44.561704 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:29:44.561704 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:29:44.562746 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:44.562746 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:44.562746 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:44.562746 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:44.562746 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:44.562746 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:44.562746 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:44.562746 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:44.562746 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:44.562746 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:44.562746 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:44.562746 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:44.562746 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:44.562746 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:44.562746 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:44.562746 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:44.562746 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:29:44.568316 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:29:44.568316 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:44.568316 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:44.568316 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 256/508] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:29:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:44.568316 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e596843-25be-438c-8883-0e952ce7cebe None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:44.579647 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:44.609871 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:44.653021 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f65edc1-b53e-4735-a3cb-54c11858d207 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:44.666031 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e596843-25be-438c-8883-0e952ce7cebe None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:44.686201 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66fdd08-b3ba-4281-9ecb-14dc17658bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:44.686818 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66fdd08-b3ba-4281-9ecb-14dc17658bd8 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:44.687092 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66fdd08-b3ba-4281-9ecb-14dc17658bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:44.687389 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66fdd08-b3ba-4281-9ecb-14dc17658bd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:44.697291 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66fdd08-b3ba-4281-9ecb-14dc17658bd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:44.699524 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66fdd08-b3ba-4281-9ecb-14dc17658bd8 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:44.700234 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde4de58-0be9-4cb1-b0f7-e392fdbdc0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:44.701084 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4de58-0be9-4cb1-b0f7-e392fdbdc0bd None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:44.701463 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4de58-0be9-4cb1-b0f7-e392fdbdc0bd None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:44.701824 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4de58-0be9-4cb1-b0f7-e392fdbdc0bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:44.715434 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-b66fdd08-b3ba-4281-9ecb-14dc17658bd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:44.715434 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:44.715434 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:44.715434 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:44.715434 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:44.715434 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:44.715434 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:44.715434 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:44.715434 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:44.715434 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:44.715434 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:44.715434 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:44.715434 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:29:44.715434 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:29:44.715434 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:29:44.715434 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:29:44.716365 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:44.716365 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:44.716365 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:44.716365 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:44.716365 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:44.716365 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:44.716365 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:44.716365 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:44.716365 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:44.716365 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:44.716365 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:44.716365 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:44.716365 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:44.716365 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:44.716365 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:44.716365 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:44.716365 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:29:44.717258 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:29:44.717258 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:44.717258 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:44.717258 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 253/509] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:29:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:44.717494 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66fdd08-b3ba-4281-9ecb-14dc17658bd8 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:44.772881 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde4de58-0be9-4cb1-b0f7-e392fdbdc0bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:44.774350 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde4de58-0be9-4cb1-b0f7-e392fdbdc0bd None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:44.800542 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:44.816949 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:44.823409 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66fdd08-b3ba-4281-9ecb-14dc17658bd8 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:44.856681 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ba8cf-c0e4-4c1b-8f84-c63afd3bce4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:44.857312 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ba8cf-c0e4-4c1b-8f84-c63afd3bce4b None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:44.857590 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ba8cf-c0e4-4c1b-8f84-c63afd3bce4b None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:44.857861 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ba8cf-c0e4-4c1b-8f84-c63afd3bce4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:44.868094 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592ba8cf-c0e4-4c1b-8f84-c63afd3bce4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:44.869486 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592ba8cf-c0e4-4c1b-8f84-c63afd3bce4b None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:44.873530 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 257/510] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:29:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:29:44.874367 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde4de58-0be9-4cb1-b0f7-e392fdbdc0bd None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:44.886250 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-592ba8cf-c0e4-4c1b-8f84-c63afd3bce4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:44.886250 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:44.886250 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:44.886250 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:44.886250 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:44.886250 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:44.886250 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:44.886250 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:44.886250 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:44.886250 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:44.886250 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:44.886250 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:44.886250 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:29:44.886250 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:29:44.886250 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:29:44.886250 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:29:44.887463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:44.887463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:44.887463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:44.887463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:44.887463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:44.887463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:44.887463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:44.887463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:44.887463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:44.887463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:44.887463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:44.887463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:44.887463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:44.887463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:44.887463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:44.887463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:44.887463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:29:44.888308 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:29:44.888308 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:44.888308 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:44.888308 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 254/511] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:29:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:44.888308 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ba8cf-c0e4-4c1b-8f84-c63afd3bce4b None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:44.974249 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde4de58-0be9-4cb1-b0f7-e392fdbdc0bd None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:44.987925 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ba8cf-c0e4-4c1b-8f84-c63afd3bce4b None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:45.009561 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f1c31-7183-4909-8d1a-6f181aa4a37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:45.010176 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f1c31-7183-4909-8d1a-6f181aa4a37c None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:45.010451 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f1c31-7183-4909-8d1a-6f181aa4a37c None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:45.010718 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f1c31-7183-4909-8d1a-6f181aa4a37c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:45.016322 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:45.021009 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045f1c31-7183-4909-8d1a-6f181aa4a37c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:45.022304 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc0ef4-d4f4-4441-9837-003eb1883a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:45.022452 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045f1c31-7183-4909-8d1a-6f181aa4a37c None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:45.022903 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc0ef4-d4f4-4441-9837-003eb1883a21 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:45.023183 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc0ef4-d4f4-4441-9837-003eb1883a21 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:45.023487 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc0ef4-d4f4-4441-9837-003eb1883a21 None admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:45.024122 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc0ef4-d4f4-4441-9837-003eb1883a21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:45.024223 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:45.024419 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc0ef4-d4f4-4441-9837-003eb1883a21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:45.025767 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc0ef4-d4f4-4441-9837-003eb1883a21 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:45.037761 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 255/512] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:29:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:45.038496 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-045f1c31-7183-4909-8d1a-6f181aa4a37c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:45.038496 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:45.038496 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:45.038496 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:45.038496 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:45.038496 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:45.038496 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:45.038496 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:45.038496 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:45.038496 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:45.038496 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:45.038496 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:45.038496 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:29:45.038496 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:29:45.038496 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:29:45.038496 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:29:45.039422 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:45.039422 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:45.039422 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:45.039422 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:45.039422 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:45.039422 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:45.039422 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:45.039422 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:45.039422 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:45.039422 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:45.039422 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:45.039422 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:45.039422 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:45.039422 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:45.039422 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:45.039422 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:45.039422 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:29:45.040261 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:29:45.040261 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:45.040261 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:45.040420 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc0ef4-d4f4-4441-9837-003eb1883a21 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:45.041038 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 258/513] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:29:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:45.042044 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f1c31-7183-4909-8d1a-6f181aa4a37c None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:45.140179 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc0ef4-d4f4-4441-9837-003eb1883a21 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:45.143636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f1c31-7183-4909-8d1a-6f181aa4a37c None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:45.174112 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c04955b-cc5c-401e-a5e0-d9a1cbcfacf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:45.174718 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04955b-cc5c-401e-a5e0-d9a1cbcfacf0 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:45.174999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04955b-cc5c-401e-a5e0-d9a1cbcfacf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:45.175306 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04955b-cc5c-401e-a5e0-d9a1cbcfacf0 None admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:45.175949 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c04955b-cc5c-401e-a5e0-d9a1cbcfacf0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:45.176254 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c04955b-cc5c-401e-a5e0-d9a1cbcfacf0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:45.177578 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b94fd8b-e4f7-468d-a651-d80733f65687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:45.177689 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c04955b-cc5c-401e-a5e0-d9a1cbcfacf0 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:45.178204 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b94fd8b-e4f7-468d-a651-d80733f65687 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:45.178487 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b94fd8b-e4f7-468d-a651-d80733f65687 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:45.178763 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b94fd8b-e4f7-468d-a651-d80733f65687 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:45.189166 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 256/514] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:29:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:45.189961 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c04955b-cc5c-401e-a5e0-d9a1cbcfacf0 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:45.190403 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b94fd8b-e4f7-468d-a651-d80733f65687 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:45.191906 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b94fd8b-e4f7-468d-a651-d80733f65687 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:45.206658 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-7b94fd8b-e4f7-468d-a651-d80733f65687 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:45.206658 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:45.206658 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:45.206658 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:45.206658 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:45.206658 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:45.206658 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:45.206658 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:45.206658 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:45.206658 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:45.206658 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:45.206658 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:45.206658 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:29:45.206658 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:29:45.206658 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:29:45.206658 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:29:45.207689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:45.207689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:45.207689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:45.207689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:45.207689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:45.207689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:45.207689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:45.207689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:45.207689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:45.207689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:45.207689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:45.207689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:45.207689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:45.207689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:45.207689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:45.207689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:45.207689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:29:45.208555 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:29:45.208555 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:45.208555 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:45.208555 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 259/515] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:29:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:45.208555 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b94fd8b-e4f7-468d-a651-d80733f65687 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:45.243928 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:45.256448 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:45.294563 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c04955b-cc5c-401e-a5e0-d9a1cbcfacf0 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:45.311067 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b94fd8b-e4f7-468d-a651-d80733f65687 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:45.328761 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e349ef03-7a08-4733-b3cb-57c6dc7b4f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:45.329364 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349ef03-7a08-4733-b3cb-57c6dc7b4f7d None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:45.329642 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349ef03-7a08-4733-b3cb-57c6dc7b4f7d None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:45.329920 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349ef03-7a08-4733-b3cb-57c6dc7b4f7d None admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:45.330552 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e349ef03-7a08-4733-b3cb-57c6dc7b4f7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:45.330849 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e349ef03-7a08-4733-b3cb-57c6dc7b4f7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:45.332266 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e349ef03-7a08-4733-b3cb-57c6dc7b4f7d None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:45.344533 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 257/516] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:29:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:45.345165 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b6e6f-2f3b-4af1-ab6f-e076b7d03933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:45.345369 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e349ef03-7a08-4733-b3cb-57c6dc7b4f7d None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:45.345784 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b6e6f-2f3b-4af1-ab6f-e076b7d03933 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:45.346063 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b6e6f-2f3b-4af1-ab6f-e076b7d03933 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:45.346343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b6e6f-2f3b-4af1-ab6f-e076b7d03933 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:45.356932 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956b6e6f-2f3b-4af1-ab6f-e076b7d03933 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:45.358360 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956b6e6f-2f3b-4af1-ab6f-e076b7d03933 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:45.369148 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-956b6e6f-2f3b-4af1-ab6f-e076b7d03933 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 14:29:45.369148 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:45.369148 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:45.369148 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:45.369148 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:45.369148 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:45.369148 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:45.369148 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:45.369148 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:45.369148 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:45.369148 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:45.369148 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:45.369148 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:29:45.369148 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:29:45.369148 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:29:45.369148 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:29:45.370050 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:45.370050 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:45.370050 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:29:45.370050 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:29:45.370050 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:29:45.370050 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:29:45.370050 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:45.370050 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:45.370050 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:45.370050 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:45.370050 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:45.370050 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:45.370050 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:45.370050 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:45.370050 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:45.370050 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:45.370050 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:45.370050 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:45.371173 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:45.371173 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:45.371173 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:29:45.371173 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:29:45.371173 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:29:45.371173 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:29:45.371173 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:29:45.371173 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:29:45.371173 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 14:29:45.371173 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:45.371173 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 260/517] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 30 14:29:45 2022] GET /identity/v3/users/neutron => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:45.371173 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b6e6f-2f3b-4af1-ab6f-e076b7d03933 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:45.450644 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e349ef03-7a08-4733-b3cb-57c6dc7b4f7d None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:45.460956 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:45.472566 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b6e6f-2f3b-4af1-ab6f-e076b7d03933 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:45.479378 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:45.485979 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ef93b-014a-4104-9503-e860aa584b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:45.486589 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ef93b-014a-4104-9503-e860aa584b8e None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:45.486870 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ef93b-014a-4104-9503-e860aa584b8e None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:45.487143 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368ef93b-014a-4104-9503-e860aa584b8e None admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:45.487813 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368ef93b-014a-4104-9503-e860aa584b8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:45.488113 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368ef93b-014a-4104-9503-e860aa584b8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:45.489487 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368ef93b-014a-4104-9503-e860aa584b8e None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:45.500401 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 258/518] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:29:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:45.501188 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ef93b-014a-4104-9503-e860aa584b8e None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:45.506487 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c18add2-f17d-46e6-b671-56b3f7e5fd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:45.507112 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18add2-f17d-46e6-b671-56b3f7e5fd48 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:45.507426 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18add2-f17d-46e6-b671-56b3f7e5fd48 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:45.507697 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18add2-f17d-46e6-b671-56b3f7e5fd48 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:45.515957 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c18add2-f17d-46e6-b671-56b3f7e5fd48 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:45.517398 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c18add2-f17d-46e6-b671-56b3f7e5fd48 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:45.529197 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-5c18add2-f17d-46e6-b671-56b3f7e5fd48 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 14:29:45.529197 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:45.529197 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:45.529197 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:45.529197 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:45.529197 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:45.529197 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:45.529197 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:45.529197 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:45.529197 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:45.529197 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:45.529197 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:45.529197 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:29:45.529197 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:29:45.529197 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:29:45.529197 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:29:45.530109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:45.530109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:45.530109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:29:45.530109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:29:45.530109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:29:45.530109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:29:45.530109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:45.530109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:45.530109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:45.530109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:45.530109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:45.530109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:45.530109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:45.530109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:45.530109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:45.530109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:45.530109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:45.530109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:45.530982 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:45.530982 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:45.530982 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:29:45.530982 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:29:45.530982 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:29:45.530982 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:29:45.530982 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:29:45.530982 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:29:45.530982 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 14:29:45.530982 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:45.530982 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 261/519] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:29:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:45.531577 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c18add2-f17d-46e6-b671-56b3f7e5fd48 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:45.605032 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368ef93b-014a-4104-9503-e860aa584b8e None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:45.629358 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c18add2-f17d-46e6-b671-56b3f7e5fd48 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:45.639893 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cf9939-ad06-4498-a04d-d2920b50f8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:45.640577 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf9939-ad06-4498-a04d-d2920b50f8a6 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:45.640864 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf9939-ad06-4498-a04d-d2920b50f8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:45.641145 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf9939-ad06-4498-a04d-d2920b50f8a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:45.641783 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cf9939-ad06-4498-a04d-d2920b50f8a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:45.642089 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cf9939-ad06-4498-a04d-d2920b50f8a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:45.643492 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cf9939-ad06-4498-a04d-d2920b50f8a6 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:45.655888 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 259/520] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:29:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:45.656679 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cf9939-ad06-4498-a04d-d2920b50f8a6 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:45.663118 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0302b2a-2948-4941-95f0-c7a9b3b0b2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:45.663758 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0302b2a-2948-4941-95f0-c7a9b3b0b2f2 None admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:45.664038 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0302b2a-2948-4941-95f0-c7a9b3b0b2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:45.664308 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0302b2a-2948-4941-95f0-c7a9b3b0b2f2 None admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:45.664949 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0302b2a-2948-4941-95f0-c7a9b3b0b2f2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:45.666366 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0302b2a-2948-4941-95f0-c7a9b3b0b2f2 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:45.687460 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:45.687632 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 262/521] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 30 14:29:45 2022] POST /identity/v3/users => generated 247 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:29:45.688351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0302b2a-2948-4941-95f0-c7a9b3b0b2f2 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:45.716684 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:45.785743 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cf9939-ad06-4498-a04d-d2920b50f8a6 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:45.805112 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0302b2a-2948-4941-95f0-c7a9b3b0b2f2 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:45.824130 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151056f0-340f-4ff8-9ed5-695e3e7c19fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:45.824761 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151056f0-340f-4ff8-9ed5-695e3e7c19fb None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:45.825043 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151056f0-340f-4ff8-9ed5-695e3e7c19fb None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:45.825312 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151056f0-340f-4ff8-9ed5-695e3e7c19fb None admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:45.826045 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151056f0-340f-4ff8-9ed5-695e3e7c19fb None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:45.826354 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151056f0-340f-4ff8-9ed5-695e3e7c19fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:45.827808 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151056f0-340f-4ff8-9ed5-695e3e7c19fb None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:45.847857 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98747b7d-e328-45a9-8861-1d9b9dc8b5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:45.848750 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98747b7d-e328-45a9-8861-1d9b9dc8b5c0 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:45.848857 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 260/522] 10.222.0.30 () {56 vars in 1139 bytes} [Tue Aug 30 14:29:45 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:45.849252 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98747b7d-e328-45a9-8861-1d9b9dc8b5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:45.849531 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98747b7d-e328-45a9-8861-1d9b9dc8b5c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:45.849692 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151056f0-340f-4ff8-9ed5-695e3e7c19fb None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:45.858183 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98747b7d-e328-45a9-8861-1d9b9dc8b5c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:45.859687 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98747b7d-e328-45a9-8861-1d9b9dc8b5c0 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:45.872343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-98747b7d-e328-45a9-8861-1d9b9dc8b5c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:29:45.872343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:45.872343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:45.872343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:45.872343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:45.872343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:45.872343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:45.872343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:45.872343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:45.872343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:45.872343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:45.872343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:45.872343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:29:45.872343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:29:45.872343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:29:45.873226 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:29:45.873226 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:45.873226 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:45.873226 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:45.873226 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:45.873226 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:45.873226 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:45.873226 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:45.873226 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:45.873226 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:45.873226 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:45.873226 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:45.873226 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:45.873226 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:45.873226 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:45.873226 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:45.873226 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:45.874074 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:29:45.874074 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:29:45.874074 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:29:45.874074 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:29:45.874074 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:29:45.874074 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:29:45.874074 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:29:45.874074 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:45.874074 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 263/523] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:29:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:45.874700 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98747b7d-e328-45a9-8861-1d9b9dc8b5c0 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:45.951680 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:45.965272 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151056f0-340f-4ff8-9ed5-695e3e7c19fb None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:45.973743 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:46.001210 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98747b7d-e328-45a9-8861-1d9b9dc8b5c0 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:46.002736 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c66ca11-6e73-4fc3-9e45-22548db2cd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:46.003382 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66ca11-6e73-4fc3-9e45-22548db2cd2c None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:46.003672 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66ca11-6e73-4fc3-9e45-22548db2cd2c None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:46.003943 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66ca11-6e73-4fc3-9e45-22548db2cd2c None admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:46.004680 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c66ca11-6e73-4fc3-9e45-22548db2cd2c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:46.004986 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c66ca11-6e73-4fc3-9e45-22548db2cd2c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:46.006413 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c66ca11-6e73-4fc3-9e45-22548db2cd2c None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:46.023971 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 261/524] 10.222.0.30 () {56 vars in 1137 bytes} [Tue Aug 30 14:29:45 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:46.024780 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c66ca11-6e73-4fc3-9e45-22548db2cd2c None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:46.044480 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2dd93-2946-4306-8862-1c41b3003813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:46.045128 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2dd93-2946-4306-8862-1c41b3003813 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:46.045403 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2dd93-2946-4306-8862-1c41b3003813 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:46.045673 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2dd93-2946-4306-8862-1c41b3003813 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:46.052715 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b2dd93-2946-4306-8862-1c41b3003813 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:46.054158 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b2dd93-2946-4306-8862-1c41b3003813 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:46.066228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-05b2dd93-2946-4306-8862-1c41b3003813 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:29:46.066228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:46.066228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:46.066228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:46.066228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:46.066228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:46.066228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:46.066228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:46.066228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:46.066228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:46.066228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:46.066228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:46.066228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:29:46.066228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:29:46.066228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:29:46.067094 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:29:46.067094 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:46.067094 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:46.067094 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:46.067094 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:46.067094 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:46.067094 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:46.067094 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:46.067094 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:46.067094 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:46.067094 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:46.067094 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:46.067094 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:46.067094 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:46.067094 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:46.067094 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:46.067094 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:46.068702 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:29:46.068702 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:29:46.068702 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:29:46.068702 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:29:46.068702 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:29:46.068702 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:29:46.068702 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:29:46.068702 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:46.071514 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 264/525] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:29:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:46.072845 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2dd93-2946-4306-8862-1c41b3003813 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:46.132324 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c66ca11-6e73-4fc3-9e45-22548db2cd2c None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:46.171235 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33948409-0a74-47e1-ae01-5d3c0e2a97eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:46.172008 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33948409-0a74-47e1-ae01-5d3c0e2a97eb None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:46.172365 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33948409-0a74-47e1-ae01-5d3c0e2a97eb None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:46.172748 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33948409-0a74-47e1-ae01-5d3c0e2a97eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:46.182017 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2dd93-2946-4306-8862-1c41b3003813 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:46.188912 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:46.222840 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8352daa-a4a0-4cdd-92b9-c3dde584ad43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:46.223857 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8352daa-a4a0-4cdd-92b9-c3dde584ad43 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:46.224085 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8352daa-a4a0-4cdd-92b9-c3dde584ad43 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:46.224520 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8352daa-a4a0-4cdd-92b9-c3dde584ad43 None admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:46.225216 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8352daa-a4a0-4cdd-92b9-c3dde584ad43 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:46.225522 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8352daa-a4a0-4cdd-92b9-c3dde584ad43 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:46.227047 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8352daa-a4a0-4cdd-92b9-c3dde584ad43 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:46.250661 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 265/526] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 30 14:29:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:46.250661 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8352daa-a4a0-4cdd-92b9-c3dde584ad43 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:46.250661 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:46.256032 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33948409-0a74-47e1-ae01-5d3c0e2a97eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:46.257566 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33948409-0a74-47e1-ae01-5d3c0e2a97eb None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:46.358024 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8352daa-a4a0-4cdd-92b9-c3dde584ad43 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:46.367968 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 262/527] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:29:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:29:46.369019 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33948409-0a74-47e1-ae01-5d3c0e2a97eb None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:46.407936 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbee16c1-62e3-487d-9196-cafe836f4720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:46.408799 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee16c1-62e3-487d-9196-cafe836f4720 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:46.409082 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee16c1-62e3-487d-9196-cafe836f4720 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:46.409360 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee16c1-62e3-487d-9196-cafe836f4720 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:46.426972 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:46.479338 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbee16c1-62e3-487d-9196-cafe836f4720 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:46.480899 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbee16c1-62e3-487d-9196-cafe836f4720 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:46.482502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33948409-0a74-47e1-ae01-5d3c0e2a97eb None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:46.482880 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:46.517609 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e4ac73-91d1-4677-97ba-be895b67c91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:46.519911 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e4ac73-91d1-4677-97ba-be895b67c91e None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:46.520254 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e4ac73-91d1-4677-97ba-be895b67c91e None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:46.520628 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e4ac73-91d1-4677-97ba-be895b67c91e None admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:46.521382 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e4ac73-91d1-4677-97ba-be895b67c91e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:46.521756 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e4ac73-91d1-4677-97ba-be895b67c91e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:46.523179 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e4ac73-91d1-4677-97ba-be895b67c91e None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:46.541482 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 263/528] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 30 14:29:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:46.542104 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e4ac73-91d1-4677-97ba-be895b67c91e None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:46.594530 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 266/529] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:29:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:29:46.594530 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbee16c1-62e3-487d-9196-cafe836f4720 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:46.649984 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e4ac73-91d1-4677-97ba-be895b67c91e None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:46.657518 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:46.686891 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e7f543-2bed-4d26-93b6-cc4317dbce7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:46.687763 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e7f543-2bed-4d26-93b6-cc4317dbce7d None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:46.688089 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e7f543-2bed-4d26-93b6-cc4317dbce7d None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:46.688314 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e7f543-2bed-4d26-93b6-cc4317dbce7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:46.689054 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e7f543-2bed-4d26-93b6-cc4317dbce7d None admin] RBAC: Adding query filter params (role.id=1316db16eac54629a283545f1601a236, scope.project.id=01cf71ca5abf4576b89d10c85af6d2de, user.id=8878de66a03044e38e507028853a0cd1) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:46.689352 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e7f543-2bed-4d26-93b6-cc4317dbce7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:46.690796 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e7f543-2bed-4d26-93b6-cc4317dbce7d None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:46.703248 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 264/530] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:29:46 2022] GET /identity/v3/role_assignments?user.id=8878de66a03044e38e507028853a0cd1&scope.project.id=01cf71ca5abf4576b89d10c85af6d2de&role.id=1316db16eac54629a283545f1601a236 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:46.703248 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e7f543-2bed-4d26-93b6-cc4317dbce7d None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:46.708579 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbee16c1-62e3-487d-9196-cafe836f4720 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:46.711614 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:46.760696 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ffb5b7-c11f-4d9f-9319-193179b8c1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:46.761867 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ffb5b7-c11f-4d9f-9319-193179b8c1ad None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:46.762631 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ffb5b7-c11f-4d9f-9319-193179b8c1ad None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:46.763179 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ffb5b7-c11f-4d9f-9319-193179b8c1ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:46.777987 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ffb5b7-c11f-4d9f-9319-193179b8c1ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:46.779791 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ffb5b7-c11f-4d9f-9319-193179b8c1ad None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:46.799752 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-82ffb5b7-c11f-4d9f-9319-193179b8c1ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:46.799752 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:46.799752 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:46.799752 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:46.799752 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:46.799752 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:46.799752 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:46.799752 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:46.799752 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:46.799752 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:46.799752 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:46.799752 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:46.799752 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:29:46.799752 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:29:46.799752 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:29:46.799752 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:29:46.800955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:46.800955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:46.800955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:46.800955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:46.800955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:46.800955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:46.800955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:46.800955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:46.800955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:46.800955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:46.800955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:46.800955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:46.800955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:46.800955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:46.800955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:46.800955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:46.800955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:29:46.802104 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:29:46.802104 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:46.802104 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:46.805535 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 267/531] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:29:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:46.806433 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ffb5b7-c11f-4d9f-9319-193179b8c1ad None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:46.815852 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e7f543-2bed-4d26-93b6-cc4317dbce7d None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:46.858822 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6636ded-4125-4a8b-b510-4d18c8cb0da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:46.859460 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6636ded-4125-4a8b-b510-4d18c8cb0da2 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:46.859736 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6636ded-4125-4a8b-b510-4d18c8cb0da2 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:46.860002 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6636ded-4125-4a8b-b510-4d18c8cb0da2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:46.860728 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6636ded-4125-4a8b-b510-4d18c8cb0da2 None admin] RBAC: Adding query filter params (role.id=1316db16eac54629a283545f1601a236, scope.project.id=01cf71ca5abf4576b89d10c85af6d2de, user.id=6889402e54424bbd8d42c7bb759cf700) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:46.861028 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6636ded-4125-4a8b-b510-4d18c8cb0da2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:46.862607 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6636ded-4125-4a8b-b510-4d18c8cb0da2 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:46.869270 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 265/532] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:29:46 2022] GET /identity/v3/role_assignments?user.id=6889402e54424bbd8d42c7bb759cf700&scope.project.id=01cf71ca5abf4576b89d10c85af6d2de&role.id=1316db16eac54629a283545f1601a236 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:46.870207 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6636ded-4125-4a8b-b510-4d18c8cb0da2 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:46.938180 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:46.955143 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ffb5b7-c11f-4d9f-9319-193179b8c1ad None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:47.032912 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3c0e34-5df0-4c1e-aea1-41dfc498d884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:47.033477 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c0e34-5df0-4c1e-aea1-41dfc498d884 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:47.033744 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c0e34-5df0-4c1e-aea1-41dfc498d884 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:47.034006 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c0e34-5df0-4c1e-aea1-41dfc498d884 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:47.056016 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3c0e34-5df0-4c1e-aea1-41dfc498d884 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:47.057818 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3c0e34-5df0-4c1e-aea1-41dfc498d884 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:47.062612 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6636ded-4125-4a8b-b510-4d18c8cb0da2 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:47.081261 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-da3c0e34-5df0-4c1e-aea1-41dfc498d884 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:47.081261 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:47.081261 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:47.081261 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:47.081261 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:47.081261 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:47.081261 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:47.081261 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:47.081261 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:47.081261 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:47.081261 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:47.081261 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:47.081261 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:29:47.081261 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:29:47.081261 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:29:47.081261 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:29:47.082228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:47.082228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:47.082228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:47.082228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:47.082228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:47.082228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:47.082228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:47.082228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:47.082228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:47.082228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:47.082228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:47.082228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:47.082228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:47.082228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:47.082228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:47.082228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:47.082228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:29:47.083068 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:29:47.083068 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:47.083068 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:47.085326 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 268/533] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:29:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:47.090775 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3c0e34-5df0-4c1e-aea1-41dfc498d884 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:47.098466 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd63ef2-fc02-4dd4-bbda-ba99388eba0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:47.099079 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd63ef2-fc02-4dd4-bbda-ba99388eba0e None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:47.099382 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd63ef2-fc02-4dd4-bbda-ba99388eba0e None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:47.099657 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd63ef2-fc02-4dd4-bbda-ba99388eba0e None admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:47.101138 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd63ef2-fc02-4dd4-bbda-ba99388eba0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:47.101445 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd63ef2-fc02-4dd4-bbda-ba99388eba0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:47.102828 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd63ef2-fc02-4dd4-bbda-ba99388eba0e None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:47.108231 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:47.124012 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 266/534] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:29:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:47.129420 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd63ef2-fc02-4dd4-bbda-ba99388eba0e None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:47.239909 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3c0e34-5df0-4c1e-aea1-41dfc498d884 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:47.277801 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a0600a-ccf8-456d-92d2-d768ad61cc5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:47.278416 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0600a-ccf8-456d-92d2-d768ad61cc5a None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:47.278698 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0600a-ccf8-456d-92d2-d768ad61cc5a None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:47.279020 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0600a-ccf8-456d-92d2-d768ad61cc5a None admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:47.279724 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a0600a-ccf8-456d-92d2-d768ad61cc5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:47.280026 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a0600a-ccf8-456d-92d2-d768ad61cc5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:47.281531 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a0600a-ccf8-456d-92d2-d768ad61cc5a None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:47.296391 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 269/535] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:29:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:47.303961 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a0600a-ccf8-456d-92d2-d768ad61cc5a None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:47.328925 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:47.336040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd63ef2-fc02-4dd4-bbda-ba99388eba0e None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:47.392485 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b70b6f-9e59-4725-bf78-e55449148a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:47.393285 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b70b6f-9e59-4725-bf78-e55449148a27 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:47.393883 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b70b6f-9e59-4725-bf78-e55449148a27 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:47.394229 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b70b6f-9e59-4725-bf78-e55449148a27 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:47.405346 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b70b6f-9e59-4725-bf78-e55449148a27 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:47.407538 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b70b6f-9e59-4725-bf78-e55449148a27 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:47.421472 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-b8b70b6f-9e59-4725-bf78-e55449148a27 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:29:47.421472 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:47.421472 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:47.421472 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:47.421472 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:47.421472 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:47.421472 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:47.421472 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:47.421472 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:47.421472 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:47.421472 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:47.421472 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:47.421472 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:29:47.421472 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:29:47.421472 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:29:47.422527 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:29:47.422527 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:47.422527 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:47.422527 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:47.422527 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:47.422527 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:47.422527 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:47.422527 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:47.422527 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:47.422527 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:47.422527 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:47.422527 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:47.422527 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:47.422527 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:47.422527 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:47.422527 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:47.422527 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:47.423605 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:29:47.423605 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:29:47.423605 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:29:47.423605 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:29:47.423605 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:29:47.423605 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:29:47.423605 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:29:47.423605 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:47.424881 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:47.426244 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 267/536] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:29:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:47.433157 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b70b6f-9e59-4725-bf78-e55449148a27 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:47.509148 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a0600a-ccf8-456d-92d2-d768ad61cc5a None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:47.551587 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b70b6f-9e59-4725-bf78-e55449148a27 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:47.578341 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e3e66-c4f9-4e2d-abf6-48b74ab15ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:47.580199 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e3e66-c4f9-4e2d-abf6-48b74ab15ac6 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:47.581218 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e3e66-c4f9-4e2d-abf6-48b74ab15ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:47.581917 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e3e66-c4f9-4e2d-abf6-48b74ab15ac6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:47.596828 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8e3e66-c4f9-4e2d-abf6-48b74ab15ac6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:47.598572 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8e3e66-c4f9-4e2d-abf6-48b74ab15ac6 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:47.613861 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d7cd73-610a-44f2-88de-d678bd2fc285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:47.614727 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d7cd73-610a-44f2-88de-d678bd2fc285 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:47.615335 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d7cd73-610a-44f2-88de-d678bd2fc285 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:47.615813 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d7cd73-610a-44f2-88de-d678bd2fc285 None admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:47.616717 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d7cd73-610a-44f2-88de-d678bd2fc285 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:47.617194 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d7cd73-610a-44f2-88de-d678bd2fc285 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:47.618970 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d7cd73-610a-44f2-88de-d678bd2fc285 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:47.624036 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-5c8e3e66-c4f9-4e2d-abf6-48b74ab15ac6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:29:47.624036 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:47.624036 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:47.624036 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:47.624036 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:47.624036 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:47.624036 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:47.624036 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:47.624036 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:47.624036 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:47.624036 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:47.624036 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:47.624036 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:29:47.624036 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:29:47.624036 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:29:47.624982 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:29:47.624982 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:47.624982 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:47.624982 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:47.624982 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:47.624982 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:47.624982 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:47.624982 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:47.624982 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:47.624982 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:47.624982 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:47.624982 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:47.624982 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:47.624982 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:47.624982 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:47.624982 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:47.624982 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:47.625831 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:29:47.625831 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:29:47.625831 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:29:47.625831 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:29:47.625831 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:29:47.625831 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:29:47.625831 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:29:47.625831 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:47.631707 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 270/537] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:29:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:29:47.634997 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 268/538] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 30 14:29:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:47.639253 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e3e66-c4f9-4e2d-abf6-48b74ab15ac6 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:47.641876 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d7cd73-610a-44f2-88de-d678bd2fc285 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:47.675616 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:47.761760 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d7cd73-610a-44f2-88de-d678bd2fc285 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:47.809619 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79fc85f-4067-4ba5-aa12-d7f25341307d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:47.810516 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79fc85f-4067-4ba5-aa12-d7f25341307d None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:47.810972 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79fc85f-4067-4ba5-aa12-d7f25341307d None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:47.811243 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e3e66-c4f9-4e2d-abf6-48b74ab15ac6 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:47.811647 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79fc85f-4067-4ba5-aa12-d7f25341307d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:47.812805 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79fc85f-4067-4ba5-aa12-d7f25341307d None admin] RBAC: Adding query filter params (role.id=1316db16eac54629a283545f1601a236, scope.project.id=01cf71ca5abf4576b89d10c85af6d2de, user.id=9e14e86e085d4987aece69fe64867ce6) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:47.813291 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79fc85f-4067-4ba5-aa12-d7f25341307d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:47.815774 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79fc85f-4067-4ba5-aa12-d7f25341307d None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:47.831235 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 269/539] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:29:47 2022] GET /identity/v3/role_assignments?user.id=9e14e86e085d4987aece69fe64867ce6&scope.project.id=01cf71ca5abf4576b89d10c85af6d2de&role.id=1316db16eac54629a283545f1601a236 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:29:47.831235 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79913ad6-7810-4e49-8284-c67173af0028 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:47.831235 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79913ad6-7810-4e49-8284-c67173af0028 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:47.831235 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79913ad6-7810-4e49-8284-c67173af0028 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:47.831235 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 270/540] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:47.839989 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82642620-3e1d-4669-834f-3e579bfbd34d None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:47.840356 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82642620-3e1d-4669-834f-3e579bfbd34d None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:47.841489 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82642620-3e1d-4669-834f-3e579bfbd34d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:47.864447 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:47.875310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-82642620-3e1d-4669-834f-3e579bfbd34d None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:47.876759 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-82642620-3e1d-4669-834f-3e579bfbd34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:47.886462 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d74233-f947-40c8-b48f-ecf68a168422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:47.887921 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d74233-f947-40c8-b48f-ecf68a168422 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:47.888435 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d74233-f947-40c8-b48f-ecf68a168422 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:47.888904 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d74233-f947-40c8-b48f-ecf68a168422 None admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:47.889826 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d74233-f947-40c8-b48f-ecf68a168422 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:47.890335 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d74233-f947-40c8-b48f-ecf68a168422 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:47.892085 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d74233-f947-40c8-b48f-ecf68a168422 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:47.908636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 271/541] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 30 14:29:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:47.915143 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d74233-f947-40c8-b48f-ecf68a168422 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:48.018559 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 271/542] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:48.085481 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d74233-f947-40c8-b48f-ecf68a168422 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:48.122650 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ffdbcc-f208-4b10-9cad-73c1b732fbbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:48.123705 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ffdbcc-f208-4b10-9cad-73c1b732fbbe None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:48.124186 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ffdbcc-f208-4b10-9cad-73c1b732fbbe None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:48.124653 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ffdbcc-f208-4b10-9cad-73c1b732fbbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:48.125572 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ffdbcc-f208-4b10-9cad-73c1b732fbbe None admin] RBAC: Adding query filter params (role.id=1316db16eac54629a283545f1601a236, scope.project.id=01cf71ca5abf4576b89d10c85af6d2de, user.id=6fda7075198d48da86aa9c0dab6ff7ca) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:48.126097 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ffdbcc-f208-4b10-9cad-73c1b732fbbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:48.128339 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ffdbcc-f208-4b10-9cad-73c1b732fbbe None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:48.140367 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 272/543] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:29:47 2022] GET /identity/v3/role_assignments?user.id=6fda7075198d48da86aa9c0dab6ff7ca&scope.project.id=01cf71ca5abf4576b89d10c85af6d2de&role.id=1316db16eac54629a283545f1601a236 => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:48.348377 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82642620-3e1d-4669-834f-3e579bfbd34d None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:48.360952 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:48.468421 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82642620-3e1d-4669-834f-3e579bfbd34d None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:48.515356 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd56b24a-5cec-46c4-b3a5-a2448485d163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:48.516343 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd56b24a-5cec-46c4-b3a5-a2448485d163 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:48.516821 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd56b24a-5cec-46c4-b3a5-a2448485d163 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:48.517284 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd56b24a-5cec-46c4-b3a5-a2448485d163 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:48.528792 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd56b24a-5cec-46c4-b3a5-a2448485d163 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:48.533908 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-fd56b24a-5cec-46c4-b3a5-a2448485d163 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:29:48.533908 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:48.533908 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:48.533908 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:48.533908 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:48.533908 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:48.533908 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:48.533908 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:48.533908 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:48.533908 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:48.533908 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:48.533908 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:48.533908 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:48.533908 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:48.533908 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:48.533908 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:48.534777 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:48.534777 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:48.534777 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:48.534777 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:48.534777 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:48.534777 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:48.534777 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:48.534777 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:48.534777 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:48.534777 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:48.534777 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:48.534777 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:48.534777 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:48.534777 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:48.534777 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:48.534777 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:48.535603 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:48.535603 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:48.535603 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:48.535603 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:48.535603 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:48.535603 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:48.535603 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:48.535603 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:48.535603 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:48.535603 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:48.535603 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:29:48.535603 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:48.538161 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 272/544] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 30 14:29:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:48.544307 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ffdbcc-f208-4b10-9cad-73c1b732fbbe None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:48.598729 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:48.652343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ffdbcc-f208-4b10-9cad-73c1b732fbbe None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:48.688845 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c3eee-4362-4f1d-bc1f-1adae59d4d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:48.689462 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c3eee-4362-4f1d-bc1f-1adae59d4d02 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:48.689785 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c3eee-4362-4f1d-bc1f-1adae59d4d02 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:48.690064 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c3eee-4362-4f1d-bc1f-1adae59d4d02 None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:48.690733 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2c3eee-4362-4f1d-bc1f-1adae59d4d02 None admin] RBAC: Adding query filter params (name=service) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:48.691035 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2c3eee-4362-4f1d-bc1f-1adae59d4d02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:48.692451 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2c3eee-4362-4f1d-bc1f-1adae59d4d02 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:48.701285 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 273/545] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 30 14:29:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:48.706896 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd56b24a-5cec-46c4-b3a5-a2448485d163 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:48.786792 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:48.828439 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd56b24a-5cec-46c4-b3a5-a2448485d163 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:48.875775 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b87f3-0964-49b4-bcb9-2f288e1c0772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:48.876831 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b87f3-0964-49b4-bcb9-2f288e1c0772 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:48.877377 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b87f3-0964-49b4-bcb9-2f288e1c0772 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:48.877884 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b87f3-0964-49b4-bcb9-2f288e1c0772 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:48.895276 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80b87f3-0964-49b4-bcb9-2f288e1c0772 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:48.897083 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80b87f3-0964-49b4-bcb9-2f288e1c0772 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:48.912445 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-f80b87f3-0964-49b4-bcb9-2f288e1c0772 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:48.912445 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:48.912445 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:48.912445 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:48.912445 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:48.912445 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:48.912445 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:48.912445 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:48.912445 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:48.912445 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:48.912445 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:48.912445 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:48.912445 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:29:48.912445 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:29:48.912445 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:29:48.912445 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:29:48.913355 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:48.913355 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:48.913355 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:48.913355 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:48.913355 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:48.913355 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:48.913355 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:48.913355 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:48.913355 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:48.913355 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:48.913355 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:48.913355 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:48.913355 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:48.913355 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:48.913355 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:48.913355 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:48.913355 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:29:48.914186 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:29:48.914186 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:48.914186 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:48.916205 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 273/546] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:29:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:48.922379 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c3eee-4362-4f1d-bc1f-1adae59d4d02 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:49.029501 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c3eee-4362-4f1d-bc1f-1adae59d4d02 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:49.047556 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:49.066593 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015378f2-a5a5-447c-8d9a-788be6069ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:49.067225 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015378f2-a5a5-447c-8d9a-788be6069ee7 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:49.067515 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015378f2-a5a5-447c-8d9a-788be6069ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:49.067790 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015378f2-a5a5-447c-8d9a-788be6069ee7 None admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:49.068439 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015378f2-a5a5-447c-8d9a-788be6069ee7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:49.068741 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015378f2-a5a5-447c-8d9a-788be6069ee7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:49.070108 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015378f2-a5a5-447c-8d9a-788be6069ee7 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:49.087477 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 274/547] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:29:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:49.088906 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b87f3-0964-49b4-bcb9-2f288e1c0772 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:49.132880 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db02aa-cff3-40e0-9987-34036bd17e20 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:49.133158 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db02aa-cff3-40e0-9987-34036bd17e20 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:49.133407 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db02aa-cff3-40e0-9987-34036bd17e20 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:49.135073 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 275/548] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:49.148619 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c310b86-2822-468d-aa60-6b6183876c63 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:49.148913 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c310b86-2822-468d-aa60-6b6183876c63 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:49.149167 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c310b86-2822-468d-aa60-6b6183876c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:49.194528 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-8c310b86-2822-468d-aa60-6b6183876c63 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:49.195768 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-8c310b86-2822-468d-aa60-6b6183876c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:49.203726 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b87f3-0964-49b4-bcb9-2f288e1c0772 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:49.224185 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:49.239541 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a0996-351a-4b2c-97b4-afa71980dd71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:49.240168 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a0996-351a-4b2c-97b4-afa71980dd71 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:49.240496 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a0996-351a-4b2c-97b4-afa71980dd71 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:49.240769 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a0996-351a-4b2c-97b4-afa71980dd71 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:49.249310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401a0996-351a-4b2c-97b4-afa71980dd71 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:49.250753 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401a0996-351a-4b2c-97b4-afa71980dd71 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:49.268639 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-401a0996-351a-4b2c-97b4-afa71980dd71 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 14:29:49.268639 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:49.268639 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:49.268639 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:49.268639 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:49.268639 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:49.268639 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:49.268639 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:49.268639 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:49.268639 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:49.268639 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:49.268639 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:49.268639 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:29:49.268639 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:29:49.268639 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:29:49.268639 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:29:49.270209 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:49.270209 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:49.270209 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:29:49.270209 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:29:49.270209 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:29:49.270209 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:29:49.270209 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:49.270209 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:49.270209 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:49.270209 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:49.270209 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:49.270209 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:49.270209 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:49.270209 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:49.270209 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:49.270209 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:49.270209 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:49.270209 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:49.271286 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:49.271286 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:49.271286 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:29:49.271286 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:29:49.271286 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:29:49.271286 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:29:49.271286 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:29:49.271286 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:29:49.271286 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 14:29:49.271286 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:49.273691 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 274/549] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 30 14:29:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:49.281007 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a0996-351a-4b2c-97b4-afa71980dd71 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:49.298470 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 276/550] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:29:49.359952 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71cf3fb-8f4d-4a5d-8b20-1ea735d8ad52 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:49.360580 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71cf3fb-8f4d-4a5d-8b20-1ea735d8ad52 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:49.361055 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71cf3fb-8f4d-4a5d-8b20-1ea735d8ad52 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:49.366158 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 277/551] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:29:49.367526 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6926bd-c949-4efd-9489-b70dbc2fd55d None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:49.368033 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6926bd-c949-4efd-9489-b70dbc2fd55d None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:49.368512 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6926bd-c949-4efd-9489-b70dbc2fd55d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:49.386986 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a0996-351a-4b2c-97b4-afa71980dd71 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:49.410129 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-7a6926bd-c949-4efd-9489-b70dbc2fd55d None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:49.411341 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-7a6926bd-c949-4efd-9489-b70dbc2fd55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:49.456525 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a415c-cb68-4797-8e93-b1f29803478e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:49.457151 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a415c-cb68-4797-8e93-b1f29803478e None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:49.457435 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a415c-cb68-4797-8e93-b1f29803478e None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:49.457712 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a415c-cb68-4797-8e93-b1f29803478e None admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:49.458453 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78a415c-cb68-4797-8e93-b1f29803478e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:49.458757 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78a415c-cb68-4797-8e93-b1f29803478e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:49.460214 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78a415c-cb68-4797-8e93-b1f29803478e None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:49.489190 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 278/552] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:49.500629 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 275/553] 10.222.0.30 () {56 vars in 1135 bytes} [Tue Aug 30 14:29:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:49.508269 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6926bd-c949-4efd-9489-b70dbc2fd55d None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:49.549524 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:49.643512 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6926bd-c949-4efd-9489-b70dbc2fd55d None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:49.654840 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a415c-cb68-4797-8e93-b1f29803478e None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:49.694087 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7514a9-8232-481e-8fa1-ca948ca8f0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:49.694745 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7514a9-8232-481e-8fa1-ca948ca8f0f9 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:49.695024 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7514a9-8232-481e-8fa1-ca948ca8f0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:49.695313 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7514a9-8232-481e-8fa1-ca948ca8f0f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:49.728332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:49.839611 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7514a9-8232-481e-8fa1-ca948ca8f0f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:49.841183 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7514a9-8232-481e-8fa1-ca948ca8f0f9 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:49.853371 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a415c-cb68-4797-8e93-b1f29803478e None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:49.893246 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:49.910651 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51154bd-bc35-4f39-8617-ec340bf8c766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:49.912034 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51154bd-bc35-4f39-8617-ec340bf8c766 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:49.912800 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51154bd-bc35-4f39-8617-ec340bf8c766 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:49.913281 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51154bd-bc35-4f39-8617-ec340bf8c766 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:49.927762 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51154bd-bc35-4f39-8617-ec340bf8c766 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:49.934816 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-f51154bd-bc35-4f39-8617-ec340bf8c766 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:29:49.934816 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:49.934816 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:49.934816 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:49.934816 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:49.934816 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:49.934816 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:49.934816 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:49.934816 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:49.934816 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:49.934816 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:49.934816 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:49.934816 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:49.934816 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:49.934816 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:49.934816 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:49.935758 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:49.935758 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:49.935758 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:49.935758 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:49.935758 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:49.935758 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:49.935758 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:49.935758 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:49.935758 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:49.935758 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:49.935758 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:49.935758 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:49.935758 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:49.935758 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:49.935758 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:49.935758 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:49.936578 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:49.936578 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:49.936578 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:49.936578 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:49.936578 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:49.936578 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:49.936578 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:49.936578 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:49.936578 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:49.936578 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:49.936578 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:29:49.936578 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:49.944351 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 276/554] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 30 14:29:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:29:49.945373 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51154bd-bc35-4f39-8617-ec340bf8c766 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:49.981552 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 279/555] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:29:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:29:49.982670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7514a9-8232-481e-8fa1-ca948ca8f0f9 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:50.063905 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:50.068075 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51154bd-bc35-4f39-8617-ec340bf8c766 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:50.098353 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7514a9-8232-481e-8fa1-ca948ca8f0f9 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:50.110461 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7242815c-07df-42c4-ac65-76fb13b04852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:50.111532 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242815c-07df-42c4-ac65-76fb13b04852 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:50.111817 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242815c-07df-42c4-ac65-76fb13b04852 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:50.112088 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242815c-07df-42c4-ac65-76fb13b04852 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:50.126186 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7242815c-07df-42c4-ac65-76fb13b04852 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:50.132568 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:50.133724 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-7242815c-07df-42c4-ac65-76fb13b04852 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:29:50.133724 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:50.133724 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:50.133724 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:50.133724 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:50.133724 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:50.133724 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:50.133724 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:50.133724 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:50.133724 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:50.133724 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:50.133724 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:50.133724 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:50.133724 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:50.133724 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:50.133724 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:50.134762 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:50.134762 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:50.134762 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:50.134762 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:50.134762 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:50.134762 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:50.134762 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:50.134762 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:50.134762 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:50.134762 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:50.134762 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:50.134762 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:50.134762 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:50.134762 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:50.134762 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:50.134762 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:50.135598 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:50.135598 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:50.135598 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:50.135598 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:50.135598 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:50.135598 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:50.135598 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:50.135598 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:50.135598 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:50.135598 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:50.135598 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:29:50.135598 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:50.135598 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 277/556] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 30 14:29:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:50.137260 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7242815c-07df-42c4-ac65-76fb13b04852 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:50.139774 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad819d4-add0-49b1-8a23-d0e1d6dc5d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:50.140791 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad819d4-add0-49b1-8a23-d0e1d6dc5d38 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:50.141253 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad819d4-add0-49b1-8a23-d0e1d6dc5d38 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:50.141624 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad819d4-add0-49b1-8a23-d0e1d6dc5d38 None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:50.142637 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad819d4-add0-49b1-8a23-d0e1d6dc5d38 None admin] RBAC: Adding query filter params (name=service) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:50.143092 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad819d4-add0-49b1-8a23-d0e1d6dc5d38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:50.144605 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad819d4-add0-49b1-8a23-d0e1d6dc5d38 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:50.156317 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 280/557] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 30 14:29:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:50.157167 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad819d4-add0-49b1-8a23-d0e1d6dc5d38 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:50.264629 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7242815c-07df-42c4-ac65-76fb13b04852 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:50.282294 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad819d4-add0-49b1-8a23-d0e1d6dc5d38 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:50.308691 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a4cf1-ff74-4f65-976b-f853aacf8704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:50.309581 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a4cf1-ff74-4f65-976b-f853aacf8704 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:50.310095 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a4cf1-ff74-4f65-976b-f853aacf8704 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:50.310267 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a4cf1-ff74-4f65-976b-f853aacf8704 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:50.320667 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59a4cf1-ff74-4f65-976b-f853aacf8704 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:50.321112 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6093deec-bc50-4e76-9290-4bd72249e933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:50.321770 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093deec-bc50-4e76-9290-4bd72249e933 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:50.322059 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093deec-bc50-4e76-9290-4bd72249e933 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:50.322153 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59a4cf1-ff74-4f65-976b-f853aacf8704 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:50.322331 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093deec-bc50-4e76-9290-4bd72249e933 None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:50.322999 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6093deec-bc50-4e76-9290-4bd72249e933 None admin] RBAC: Adding query filter params (name=service) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:50.323324 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6093deec-bc50-4e76-9290-4bd72249e933 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:50.325070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6093deec-bc50-4e76-9290-4bd72249e933 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:50.333308 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 281/558] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 30 14:29:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:50.334200 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6093deec-bc50-4e76-9290-4bd72249e933 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:50.337224 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-b59a4cf1-ff74-4f65-976b-f853aacf8704 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:50.337224 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:50.337224 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:50.337224 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:50.337224 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:50.337224 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:50.337224 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:50.337224 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:50.337224 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:50.337224 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:50.337224 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:50.337224 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:50.337224 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:29:50.337224 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:29:50.337224 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:29:50.337224 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:29:50.338132 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:50.338132 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:50.338132 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:50.338132 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:50.338132 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:50.338132 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:50.338132 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:50.338132 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:50.338132 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:50.338132 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:50.338132 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:50.338132 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:50.338132 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:50.338132 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:50.338132 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:50.338132 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:50.338132 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:29:50.339041 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:29:50.339041 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:50.339041 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:50.339041 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 278/559] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:29:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:50.340184 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206151a-271b-43c8-86c3-848a12f5d1fc None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:50.341240 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206151a-271b-43c8-86c3-848a12f5d1fc None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:50.341240 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206151a-271b-43c8-86c3-848a12f5d1fc None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:50.342071 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 279/560] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:50.342627 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b206151a-271b-43c8-86c3-848a12f5d1fc None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:50.360164 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:50.406574 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:50.453698 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6093deec-bc50-4e76-9290-4bd72249e933 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:50.458628 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b206151a-271b-43c8-86c3-848a12f5d1fc None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:50.492699 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e55809-d47d-4ca2-8f49-dabd1abd746b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:50.493257 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e55809-d47d-4ca2-8f49-dabd1abd746b None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:50.493539 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e55809-d47d-4ca2-8f49-dabd1abd746b None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:50.493813 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e55809-d47d-4ca2-8f49-dabd1abd746b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:50.496899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c8f4c2-65f8-48ff-9760-96d3589931b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:50.497526 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8f4c2-65f8-48ff-9760-96d3589931b5 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:50.497801 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8f4c2-65f8-48ff-9760-96d3589931b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:50.498069 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8f4c2-65f8-48ff-9760-96d3589931b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:50.507878 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e55809-d47d-4ca2-8f49-dabd1abd746b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:50.509153 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c8f4c2-65f8-48ff-9760-96d3589931b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:50.509640 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e55809-d47d-4ca2-8f49-dabd1abd746b None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:50.510583 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c8f4c2-65f8-48ff-9760-96d3589931b5 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:50.524543 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-c3e55809-d47d-4ca2-8f49-dabd1abd746b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:50.524543 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:50.524543 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:50.524543 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:50.524543 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:50.524543 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:50.524543 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:50.524543 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:50.524543 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:50.524543 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:50.524543 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:50.524543 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:50.524543 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:29:50.524543 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:29:50.524543 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:29:50.524543 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:29:50.525506 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:50.525506 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:50.525506 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:50.525506 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:50.525506 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:50.525506 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:50.525506 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:50.525506 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:50.525506 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:50.525506 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:50.525506 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:50.525506 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:50.525506 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:50.525506 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:50.525506 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:50.525506 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:50.525506 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:29:50.531578 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:29:50.531578 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:50.531578 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:50.531578 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 282/561] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:29:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:50.531578 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e55809-d47d-4ca2-8f49-dabd1abd746b None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:50.531858 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-46c8f4c2-65f8-48ff-9760-96d3589931b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:50.531858 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:50.531858 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:50.531858 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:50.531858 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:50.531858 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:50.531858 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:50.531858 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:50.531858 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:50.531858 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:50.531858 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:50.531858 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:50.531858 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:29:50.531858 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:29:50.531858 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:29:50.531858 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:29:50.532770 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:50.532770 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:50.532770 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:50.532770 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:50.532770 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:50.532770 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:50.532770 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:50.532770 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:50.532770 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:50.532770 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:50.532770 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:50.532770 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:50.532770 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:50.532770 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:50.532770 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:50.532770 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:50.532770 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:29:50.533635 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:29:50.533635 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:50.533635 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:50.533635 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 280/562] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:29:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:50.534216 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f480be0-17c2-494d-aab6-7b82891c9d6e None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:50.534506 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f480be0-17c2-494d-aab6-7b82891c9d6e None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:50.534923 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f480be0-17c2-494d-aab6-7b82891c9d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:50.554221 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-5f480be0-17c2-494d-aab6-7b82891c9d6e None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:50.555427 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-5f480be0-17c2-494d-aab6-7b82891c9d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:50.612041 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:50.622698 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 281/563] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:50.623518 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f480be0-17c2-494d-aab6-7b82891c9d6e None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:50.637315 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e55809-d47d-4ca2-8f49-dabd1abd746b None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:50.646995 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:50.673839 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92680fc0-5088-4bd0-98ff-3016e6e1db44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:50.674444 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92680fc0-5088-4bd0-98ff-3016e6e1db44 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:50.674727 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92680fc0-5088-4bd0-98ff-3016e6e1db44 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:50.675009 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92680fc0-5088-4bd0-98ff-3016e6e1db44 None admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:50.675680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92680fc0-5088-4bd0-98ff-3016e6e1db44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:50.675989 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92680fc0-5088-4bd0-98ff-3016e6e1db44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:50.677358 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92680fc0-5088-4bd0-98ff-3016e6e1db44 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:50.691755 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 283/564] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:29:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:50.692752 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92680fc0-5088-4bd0-98ff-3016e6e1db44 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:50.751357 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f480be0-17c2-494d-aab6-7b82891c9d6e None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:50.784652 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cbc53e-fdbe-4fe2-878b-b201e28ada45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:50.785418 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cbc53e-fdbe-4fe2-878b-b201e28ada45 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:50.785788 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cbc53e-fdbe-4fe2-878b-b201e28ada45 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:50.786065 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cbc53e-fdbe-4fe2-878b-b201e28ada45 None admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:50.786699 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cbc53e-fdbe-4fe2-878b-b201e28ada45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:50.786991 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cbc53e-fdbe-4fe2-878b-b201e28ada45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:50.788367 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cbc53e-fdbe-4fe2-878b-b201e28ada45 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:50.799933 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 282/565] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:29:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:50.801086 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0377cc6f-d9b4-45c9-8204-b5dd6a36715e None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:50.801363 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0377cc6f-d9b4-45c9-8204-b5dd6a36715e None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:50.801629 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0377cc6f-d9b4-45c9-8204-b5dd6a36715e None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:50.802986 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 283/566] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:50.807666 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0377cc6f-d9b4-45c9-8204-b5dd6a36715e None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:50.814785 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92680fc0-5088-4bd0-98ff-3016e6e1db44 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:50.851817 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd152fe-09b5-4a73-a5aa-d5ac3c4588cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:50.852420 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd152fe-09b5-4a73-a5aa-d5ac3c4588cc None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:50.852704 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd152fe-09b5-4a73-a5aa-d5ac3c4588cc None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:50.852981 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd152fe-09b5-4a73-a5aa-d5ac3c4588cc None admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:50.853619 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd152fe-09b5-4a73-a5aa-d5ac3c4588cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:50.853924 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd152fe-09b5-4a73-a5aa-d5ac3c4588cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:50.856305 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd152fe-09b5-4a73-a5aa-d5ac3c4588cc None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:50.858634 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:50.876769 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 284/567] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:29:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:50.876769 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd152fe-09b5-4a73-a5aa-d5ac3c4588cc None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:50.905518 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:50.909517 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0377cc6f-d9b4-45c9-8204-b5dd6a36715e None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:50.944105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3e3394-7e8a-4cd2-815b-a0a39993f480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:50.944818 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3e3394-7e8a-4cd2-815b-a0a39993f480 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:50.945097 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3e3394-7e8a-4cd2-815b-a0a39993f480 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:50.945384 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3e3394-7e8a-4cd2-815b-a0a39993f480 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:50.952151 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3e3394-7e8a-4cd2-815b-a0a39993f480 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:50.953591 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3e3394-7e8a-4cd2-815b-a0a39993f480 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:50.964638 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-1a3e3394-7e8a-4cd2-815b-a0a39993f480 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:29:50.964638 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:50.964638 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:50.964638 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:50.964638 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:50.964638 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:50.964638 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:50.964638 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:50.964638 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:50.964638 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:50.964638 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:50.964638 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:50.964638 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:29:50.964638 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:29:50.964638 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:29:50.965740 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:29:50.965740 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:50.965740 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:50.965740 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:50.965740 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:50.965740 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:50.965740 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:50.965740 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:50.965740 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:50.965740 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:50.965740 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:50.965740 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:50.965740 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:50.965740 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:50.965740 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:50.965740 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:50.965740 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:50.966639 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:29:50.966639 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:29:50.966639 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:29:50.966639 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:29:50.966639 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:29:50.966639 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:29:50.966639 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:29:50.966639 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:50.966639 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 284/568] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:29:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:50.967123 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef7f0f-4564-4d62-b8c1-53a2c5624da3 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:50.967123 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef7f0f-4564-4d62-b8c1-53a2c5624da3 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:50.967366 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef7f0f-4564-4d62-b8c1-53a2c5624da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:50.977667 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd152fe-09b5-4a73-a5aa-d5ac3c4588cc None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:50.984898 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-a0ef7f0f-4564-4d62-b8c1-53a2c5624da3 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:50.986073 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-a0ef7f0f-4564-4d62-b8c1-53a2c5624da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:51.013178 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2391dc-9b18-4e61-91b9-fcebbf131a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:51.013733 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2391dc-9b18-4e61-91b9-fcebbf131a04 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:51.014006 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2391dc-9b18-4e61-91b9-fcebbf131a04 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:51.014272 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2391dc-9b18-4e61-91b9-fcebbf131a04 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:51.021547 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2391dc-9b18-4e61-91b9-fcebbf131a04 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:51.022966 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2391dc-9b18-4e61-91b9-fcebbf131a04 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:51.033894 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-2f2391dc-9b18-4e61-91b9-fcebbf131a04 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:29:51.033894 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:51.033894 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:51.033894 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:51.033894 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:51.033894 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:51.033894 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:51.033894 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:51.033894 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:51.033894 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:51.033894 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:51.033894 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:51.033894 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:29:51.033894 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:29:51.033894 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:29:51.033894 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:29:51.035012 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:51.035012 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:51.035012 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:29:51.035012 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:29:51.035012 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:29:51.035012 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:29:51.035012 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:51.035012 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:51.035012 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:51.035012 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:51.035012 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:51.035012 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:51.035012 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:51.035012 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:51.035012 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:51.035012 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:51.035012 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:51.035012 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:51.036206 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:51.036206 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:51.036206 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:29:51.036206 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:29:51.036206 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:29:51.036206 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:29:51.036206 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:29:51.036206 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:29:51.036206 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:29:51.036206 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:51.036206 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 285/569] 10.222.0.30 () {56 vars in 1093 bytes} [Tue Aug 30 14:29:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:51.036206 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2391dc-9b18-4e61-91b9-fcebbf131a04 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:51.056372 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 285/570] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:51.057208 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ef7f0f-4564-4d62-b8c1-53a2c5624da3 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:51.103324 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:51.135366 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:51.150705 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2391dc-9b18-4e61-91b9-fcebbf131a04 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:51.166169 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ef7f0f-4564-4d62-b8c1-53a2c5624da3 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:51.188633 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f1a860-cf06-4ea3-905f-04e2e7834cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:51.189219 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1a860-cf06-4ea3-905f-04e2e7834cef None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:51.189502 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1a860-cf06-4ea3-905f-04e2e7834cef None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:51.189795 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1a860-cf06-4ea3-905f-04e2e7834cef None admin] PATH_INFO: `/v3/users/glance` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:51.197872 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f1a860-cf06-4ea3-905f-04e2e7834cef None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:51.199908 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f1a860-cf06-4ea3-905f-04e2e7834cef None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:51.202921 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66856427-a1e7-42d6-a165-0dcdb3ce8d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:51.203596 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66856427-a1e7-42d6-a165-0dcdb3ce8d9a None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:51.203876 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66856427-a1e7-42d6-a165-0dcdb3ce8d9a None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:51.204150 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66856427-a1e7-42d6-a165-0dcdb3ce8d9a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:51.211061 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-78f1a860-cf06-4ea3-905f-04e2e7834cef None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 30 14:29:51.211061 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:51.211061 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:51.211061 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:51.211061 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:51.211061 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:51.211061 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:51.211061 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:51.211061 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:51.211061 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:51.211061 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:51.211061 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:51.211061 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:29:51.211061 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:29:51.211061 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:29:51.211061 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:29:51.212130 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:51.212130 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:51.212130 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:29:51.212130 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:29:51.212130 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:29:51.212130 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:29:51.212130 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:51.212130 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:51.212130 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:51.212130 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:51.212130 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:51.212130 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:51.212130 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:51.212130 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:51.212130 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:51.212130 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:51.212130 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:51.212130 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:51.213085 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:51.213085 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:51.213085 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:29:51.213085 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:29:51.213085 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:29:51.213085 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:29:51.213085 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:29:51.213085 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:29:51.213085 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 30 14:29:51.213085 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:51.213085 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 286/571] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:29:51 2022] GET /identity/v3/users/glance => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:51.213085 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f1a860-cf06-4ea3-905f-04e2e7834cef None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:51.227402 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66856427-a1e7-42d6-a165-0dcdb3ce8d9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:51.232651 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-66856427-a1e7-42d6-a165-0dcdb3ce8d9a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:29:51.232651 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:51.232651 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:51.232651 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:51.232651 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:51.232651 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:51.232651 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:51.232651 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:51.232651 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:51.232651 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:51.232651 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:51.232651 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:51.232651 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:51.232651 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:51.232651 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:51.232651 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:51.235275 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:51.235275 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:51.235275 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:51.235275 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:51.235275 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:51.235275 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:51.235275 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:51.235275 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:51.235275 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:51.235275 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:51.235275 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:51.235275 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:51.235275 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:51.235275 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:51.235275 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:51.235275 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:51.236184 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:51.236184 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:51.236184 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:51.236184 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:51.236184 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:51.236184 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:51.236184 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:51.236184 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:51.236184 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:51.236184 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:51.236184 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:29:51.236184 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:51.236184 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 286/572] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 30 14:29:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:51.236184 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66856427-a1e7-42d6-a165-0dcdb3ce8d9a None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:51.329683 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f1a860-cf06-4ea3-905f-04e2e7834cef None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:51.342420 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66856427-a1e7-42d6-a165-0dcdb3ce8d9a None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:51.344711 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:51.364334 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd2cd7-65cf-43a9-98ea-3a29c11388b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:51.364937 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd2cd7-65cf-43a9-98ea-3a29c11388b0 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:51.365225 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd2cd7-65cf-43a9-98ea-3a29c11388b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:51.365497 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd2cd7-65cf-43a9-98ea-3a29c11388b0 None admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:51.366187 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdd2cd7-65cf-43a9-98ea-3a29c11388b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:51.366495 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdd2cd7-65cf-43a9-98ea-3a29c11388b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:51.367963 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdd2cd7-65cf-43a9-98ea-3a29c11388b0 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:51.376479 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172aa88d-33c4-4c84-a0e2-95f3788a867d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:51.377077 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172aa88d-33c4-4c84-a0e2-95f3788a867d None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:51.377356 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172aa88d-33c4-4c84-a0e2-95f3788a867d None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:51.377618 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172aa88d-33c4-4c84-a0e2-95f3788a867d None admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:51.378334 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172aa88d-33c4-4c84-a0e2-95f3788a867d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:51.378630 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172aa88d-33c4-4c84-a0e2-95f3788a867d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:51.380048 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172aa88d-33c4-4c84-a0e2-95f3788a867d None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:51.380146 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 287/573] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 30 14:29:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:51.380912 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd2cd7-65cf-43a9-98ea-3a29c11388b0 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:51.395072 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:51.397089 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 287/574] 10.222.0.30 () {56 vars in 1133 bytes} [Tue Aug 30 14:29:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:51.397853 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172aa88d-33c4-4c84-a0e2-95f3788a867d None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:51.486057 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd2cd7-65cf-43a9-98ea-3a29c11388b0 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:51.494518 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172aa88d-33c4-4c84-a0e2-95f3788a867d None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:51.519412 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44bdbb7-13f7-4293-9294-d7193da7c6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:51.520009 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44bdbb7-13f7-4293-9294-d7193da7c6f1 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:51.520281 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44bdbb7-13f7-4293-9294-d7193da7c6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:51.520569 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44bdbb7-13f7-4293-9294-d7193da7c6f1 None admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:51.521270 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44bdbb7-13f7-4293-9294-d7193da7c6f1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:51.521568 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44bdbb7-13f7-4293-9294-d7193da7c6f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:51.522972 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44bdbb7-13f7-4293-9294-d7193da7c6f1 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:51.527919 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03390196-313f-4c0d-8bf5-c1b628ad83c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:51.528517 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03390196-313f-4c0d-8bf5-c1b628ad83c3 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:51.528793 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03390196-313f-4c0d-8bf5-c1b628ad83c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:51.529053 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03390196-313f-4c0d-8bf5-c1b628ad83c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:51.529700 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03390196-313f-4c0d-8bf5-c1b628ad83c3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:51.529995 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03390196-313f-4c0d-8bf5-c1b628ad83c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:51.531359 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03390196-313f-4c0d-8bf5-c1b628ad83c3 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:51.539073 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 288/575] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 30 14:29:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:51.545025 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03390196-313f-4c0d-8bf5-c1b628ad83c3 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:51.545190 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 288/576] 10.222.0.30 () {56 vars in 1137 bytes} [Tue Aug 30 14:29:51 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:51.545190 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44bdbb7-13f7-4293-9294-d7193da7c6f1 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:51.568870 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:51.609279 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:51.644968 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03390196-313f-4c0d-8bf5-c1b628ad83c3 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:51.646585 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44bdbb7-13f7-4293-9294-d7193da7c6f1 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:51.678204 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f0c267-f7d1-4b3c-8f90-d5e4de0f8500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:51.678824 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f0c267-f7d1-4b3c-8f90-d5e4de0f8500 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:51.679102 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f0c267-f7d1-4b3c-8f90-d5e4de0f8500 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:51.679413 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f0c267-f7d1-4b3c-8f90-d5e4de0f8500 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:51.680848 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90650fc0-59d4-423d-a301-c2b4b25983a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:51.681450 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90650fc0-59d4-423d-a301-c2b4b25983a7 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:51.681728 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90650fc0-59d4-423d-a301-c2b4b25983a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:51.682004 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90650fc0-59d4-423d-a301-c2b4b25983a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:51.682727 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90650fc0-59d4-423d-a301-c2b4b25983a7 None admin] RBAC: Adding query filter params (role.id=1316db16eac54629a283545f1601a236, scope.project.id=01cf71ca5abf4576b89d10c85af6d2de, user.id=a18b13e3708144e1b5e8df80ec3a4e83) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:51.683663 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90650fc0-59d4-423d-a301-c2b4b25983a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:51.685329 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90650fc0-59d4-423d-a301-c2b4b25983a7 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:51.690081 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f0c267-f7d1-4b3c-8f90-d5e4de0f8500 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:51.690752 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 289/577] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:29:51 2022] GET /identity/v3/role_assignments?user.id=a18b13e3708144e1b5e8df80ec3a4e83&scope.project.id=01cf71ca5abf4576b89d10c85af6d2de&role.id=1316db16eac54629a283545f1601a236 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:51.691559 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90650fc0-59d4-423d-a301-c2b4b25983a7 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:51.695359 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-b5f0c267-f7d1-4b3c-8f90-d5e4de0f8500 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:29:51.695359 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:51.695359 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:51.695359 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:51.695359 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:51.695359 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:51.695359 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:51.695359 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:51.695359 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:51.695359 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:51.695359 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:51.695359 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:51.695359 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:51.695359 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:51.695359 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:51.695359 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:51.696320 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:51.696320 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:51.696320 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:51.696320 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:51.696320 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:51.696320 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:51.696320 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:51.696320 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:51.696320 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:51.696320 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:51.696320 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:51.696320 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:51.696320 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:51.696320 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:51.696320 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:51.696320 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:51.697162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:51.697162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:51.697162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:51.697162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:51.697162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:51.697162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:51.697162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:51.697162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:51.697162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:51.697162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:51.697162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:29:51.697162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:51.702650 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 289/578] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 30 14:29:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:51.703671 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f0c267-f7d1-4b3c-8f90-d5e4de0f8500 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:51.801642 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:51.806823 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90650fc0-59d4-423d-a301-c2b4b25983a7 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:51.824212 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f0c267-f7d1-4b3c-8f90-d5e4de0f8500 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:51.840224 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:51.852416 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e90ca-5aca-482e-93f4-076d3b2b6bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:51.853232 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e90ca-5aca-482e-93f4-076d3b2b6bf2 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:51.853514 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e90ca-5aca-482e-93f4-076d3b2b6bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:51.853786 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e90ca-5aca-482e-93f4-076d3b2b6bf2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:51.866279 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b69807-ed0c-4fa7-9fb3-8f16b5af8818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:51.866852 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b69807-ed0c-4fa7-9fb3-8f16b5af8818 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:51.867132 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b69807-ed0c-4fa7-9fb3-8f16b5af8818 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:51.867441 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b69807-ed0c-4fa7-9fb3-8f16b5af8818 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:51.873128 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1e90ca-5aca-482e-93f4-076d3b2b6bf2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:51.875000 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1e90ca-5aca-482e-93f4-076d3b2b6bf2 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:51.878115 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b69807-ed0c-4fa7-9fb3-8f16b5af8818 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:51.879590 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b69807-ed0c-4fa7-9fb3-8f16b5af8818 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:51.890451 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-8d1e90ca-5aca-482e-93f4-076d3b2b6bf2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:51.890451 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:51.890451 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:51.890451 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:51.890451 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:51.890451 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:51.890451 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:51.890451 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:51.890451 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:51.890451 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:51.890451 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:51.890451 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:51.890451 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:29:51.890451 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:29:51.890451 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:29:51.890451 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:29:51.891475 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:51.891475 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:51.891475 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:51.891475 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:51.891475 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:51.891475 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:51.891475 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:51.891475 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:51.891475 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:51.891475 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:51.891475 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:51.891475 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:51.891475 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:51.891475 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:51.891475 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:51.891475 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:51.891475 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:29:51.892562 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:29:51.892562 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:51.892562 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:51.893983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 290/579] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:29:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:51.894511 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-e1b69807-ed0c-4fa7-9fb3-8f16b5af8818 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:51.894511 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:51.894511 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:51.894511 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:51.894511 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:51.894511 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:51.894511 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:51.894511 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:51.894511 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:51.894511 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:51.894511 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:51.894511 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:51.894511 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:29:51.894511 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:29:51.894511 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:29:51.894511 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:29:51.895420 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:51.895420 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:51.895420 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:51.895420 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:51.895420 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:51.895420 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:51.895420 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:51.895420 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:51.895420 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:51.895420 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:51.895420 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:51.895420 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:51.895420 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:51.895420 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:51.895420 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:51.895420 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:51.895420 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:29:51.899952 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:29:51.899952 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:51.899952 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:51.899952 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 290/580] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:29:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:51.899952 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b69807-ed0c-4fa7-9fb3-8f16b5af8818 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:51.900414 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e90ca-5aca-482e-93f4-076d3b2b6bf2 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:52.005255 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b69807-ed0c-4fa7-9fb3-8f16b5af8818 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:52.012143 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e90ca-5aca-482e-93f4-076d3b2b6bf2 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:52.048689 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e55d244-ae13-4c6b-88ad-1a465e394308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:52.049561 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e55d244-ae13-4c6b-88ad-1a465e394308 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:52.049845 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e55d244-ae13-4c6b-88ad-1a465e394308 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:52.050118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e55d244-ae13-4c6b-88ad-1a465e394308 None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:52.050692 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d2744f-eb75-46e3-af94-17d69070264a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:52.050805 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e55d244-ae13-4c6b-88ad-1a465e394308 None admin] RBAC: Adding query filter params (name=service) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:52.051074 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e55d244-ae13-4c6b-88ad-1a465e394308 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:52.051351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d2744f-eb75-46e3-af94-17d69070264a None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:52.051630 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d2744f-eb75-46e3-af94-17d69070264a None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:52.051900 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d2744f-eb75-46e3-af94-17d69070264a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:52.052498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e55d244-ae13-4c6b-88ad-1a465e394308 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:52.054063 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:52.062248 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 291/581] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 30 14:29:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:52.062897 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e55d244-ae13-4c6b-88ad-1a465e394308 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:52.069659 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d2744f-eb75-46e3-af94-17d69070264a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:52.071118 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d2744f-eb75-46e3-af94-17d69070264a None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:52.086657 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-b3d2744f-eb75-46e3-af94-17d69070264a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:52.086657 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:52.086657 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:52.086657 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:52.086657 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:52.086657 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:52.086657 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:52.086657 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:52.086657 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:52.086657 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:52.086657 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:52.086657 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:52.086657 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:29:52.086657 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:29:52.086657 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:29:52.086657 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:29:52.087687 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:52.087687 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:52.087687 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:52.087687 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:52.087687 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:52.087687 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:52.087687 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:52.087687 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:52.087687 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:52.087687 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:52.087687 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:52.087687 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:52.087687 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:52.087687 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:52.087687 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:52.087687 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:52.087687 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:29:52.088603 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:29:52.088603 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:52.088603 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:52.092670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 291/582] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:29:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:52.093519 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d2744f-eb75-46e3-af94-17d69070264a None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:52.130375 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:52.175071 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e55d244-ae13-4c6b-88ad-1a465e394308 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:52.207992 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d2744f-eb75-46e3-af94-17d69070264a None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:52.221767 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5267254c-9434-4d79-84e4-79c6c44cdced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:52.222380 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5267254c-9434-4d79-84e4-79c6c44cdced None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:52.222659 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5267254c-9434-4d79-84e4-79c6c44cdced None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:52.222932 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5267254c-9434-4d79-84e4-79c6c44cdced None admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:52.223611 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5267254c-9434-4d79-84e4-79c6c44cdced None admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:52.223920 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5267254c-9434-4d79-84e4-79c6c44cdced None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:52.225279 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5267254c-9434-4d79-84e4-79c6c44cdced None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:52.239411 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 292/583] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:29:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:52.240060 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5267254c-9434-4d79-84e4-79c6c44cdced None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:52.252655 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a7b80f-f172-436e-be15-3cca4a47dfea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:52.253259 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a7b80f-f172-436e-be15-3cca4a47dfea None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:52.253603 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a7b80f-f172-436e-be15-3cca4a47dfea None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:52.253876 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a7b80f-f172-436e-be15-3cca4a47dfea None admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:52.254538 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a7b80f-f172-436e-be15-3cca4a47dfea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:52.254838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a7b80f-f172-436e-be15-3cca4a47dfea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:52.256254 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a7b80f-f172-436e-be15-3cca4a47dfea None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:52.267911 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 292/584] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:29:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:52.272384 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a7b80f-f172-436e-be15-3cca4a47dfea None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:52.307485 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:52.355470 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5267254c-9434-4d79-84e4-79c6c44cdced None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:52.375491 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:52.383284 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a7b80f-f172-436e-be15-3cca4a47dfea None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:52.392015 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7943e5-0990-4d07-9522-40e8c2846ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:52.392673 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7943e5-0990-4d07-9522-40e8c2846ba1 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:52.392923 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7943e5-0990-4d07-9522-40e8c2846ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:52.393194 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7943e5-0990-4d07-9522-40e8c2846ba1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:52.403073 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7943e5-0990-4d07-9522-40e8c2846ba1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:52.404530 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7943e5-0990-4d07-9522-40e8c2846ba1 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:52.419362 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-3b7943e5-0990-4d07-9522-40e8c2846ba1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:52.419362 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:52.419362 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:52.419362 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:52.419362 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:52.419362 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:52.419362 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:52.419362 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:52.419362 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:52.419362 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:52.419362 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:52.419362 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:52.419362 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:29:52.419362 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:29:52.419362 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:29:52.419362 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:29:52.420477 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:52.420477 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:52.420477 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:52.420477 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:52.420477 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:52.420477 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:52.420477 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:52.420477 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:52.420477 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:52.420477 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:52.420477 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:52.420477 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:52.420477 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:52.420477 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:52.420477 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:52.420477 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:52.420477 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:29:52.421366 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:29:52.421366 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:52.421366 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:52.421366 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 293/585] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:29:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:52.421366 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7943e5-0990-4d07-9522-40e8c2846ba1 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:52.426532 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de82b920-b031-4d00-9b46-f2d8be87c162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:52.427145 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de82b920-b031-4d00-9b46-f2d8be87c162 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:52.427455 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de82b920-b031-4d00-9b46-f2d8be87c162 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:52.427730 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de82b920-b031-4d00-9b46-f2d8be87c162 None admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:52.428376 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de82b920-b031-4d00-9b46-f2d8be87c162 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:52.428677 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de82b920-b031-4d00-9b46-f2d8be87c162 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:52.430037 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de82b920-b031-4d00-9b46-f2d8be87c162 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:52.443639 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 293/586] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:29:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:52.444794 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de82b920-b031-4d00-9b46-f2d8be87c162 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:52.528985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7943e5-0990-4d07-9522-40e8c2846ba1 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:52.538228 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:52.550503 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de82b920-b031-4d00-9b46-f2d8be87c162 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:52.566293 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a549aed-b6e1-4c92-9417-df3f57f8cdbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:52.566924 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a549aed-b6e1-4c92-9417-df3f57f8cdbc None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:52.567202 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a549aed-b6e1-4c92-9417-df3f57f8cdbc None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:52.567516 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a549aed-b6e1-4c92-9417-df3f57f8cdbc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:52.574470 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a549aed-b6e1-4c92-9417-df3f57f8cdbc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:52.575956 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a549aed-b6e1-4c92-9417-df3f57f8cdbc None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:52.586145 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a7e31b-76cf-40da-9d96-26ddbc8846d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:52.586771 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a7e31b-76cf-40da-9d96-26ddbc8846d1 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:52.587047 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a7e31b-76cf-40da-9d96-26ddbc8846d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:52.587343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a7e31b-76cf-40da-9d96-26ddbc8846d1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:52.587662 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-4a549aed-b6e1-4c92-9417-df3f57f8cdbc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:29:52.587662 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:52.587662 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:52.587662 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:52.587662 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:52.587662 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:52.587662 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:52.587662 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:52.587662 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:52.587662 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:52.587662 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:52.587662 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:52.587662 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:29:52.587662 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:29:52.587662 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:29:52.588502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:29:52.588502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:52.588502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:52.588502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:52.588502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:52.588502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:52.588502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:52.588502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:52.588502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:52.588502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:52.588502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:52.588502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:52.588502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:52.588502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:52.588502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:52.588502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:52.588502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:52.589939 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:29:52.589939 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:29:52.589939 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:29:52.589939 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:29:52.589939 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:29:52.589939 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:29:52.589939 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:29:52.589939 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:52.589939 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 294/587] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:29:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:52.590435 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a549aed-b6e1-4c92-9417-df3f57f8cdbc None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:52.601249 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a7e31b-76cf-40da-9d96-26ddbc8846d1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:52.602687 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a7e31b-76cf-40da-9d96-26ddbc8846d1 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:52.614597 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-32a7e31b-76cf-40da-9d96-26ddbc8846d1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 14:29:52.614597 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:52.614597 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:52.614597 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:52.614597 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:52.614597 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:52.614597 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:52.614597 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:52.614597 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:52.614597 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:52.614597 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:52.614597 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:52.614597 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:29:52.614597 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:29:52.614597 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:29:52.614597 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:29:52.615618 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:52.615618 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:52.615618 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:29:52.615618 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:29:52.615618 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:29:52.615618 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:29:52.615618 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:52.615618 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:52.615618 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:52.615618 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:52.615618 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:52.615618 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:52.615618 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:52.615618 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:52.615618 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:52.615618 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:52.615618 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:52.615618 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:52.616566 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:52.616566 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:52.616566 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:29:52.616566 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:29:52.616566 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:29:52.616566 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:29:52.616566 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:29:52.616566 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:29:52.616566 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 14:29:52.616566 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:52.616566 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 294/588] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 30 14:29:52 2022] GET /identity/v3/users/neutron => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:52.617236 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a7e31b-76cf-40da-9d96-26ddbc8846d1 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:52.633246 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:52.707974 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a549aed-b6e1-4c92-9417-df3f57f8cdbc None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:52.725773 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a7e31b-76cf-40da-9d96-26ddbc8846d1 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:52.747295 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2829e941-b047-4aad-8622-8f98af1d6fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:52.747899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2829e941-b047-4aad-8622-8f98af1d6fbd None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:52.748171 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2829e941-b047-4aad-8622-8f98af1d6fbd None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:52.748433 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2829e941-b047-4aad-8622-8f98af1d6fbd None admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:52.749063 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2829e941-b047-4aad-8622-8f98af1d6fbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:52.749360 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2829e941-b047-4aad-8622-8f98af1d6fbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:52.750749 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2829e941-b047-4aad-8622-8f98af1d6fbd None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:52.762292 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def8a8a1-77f0-41a3-b0a9-3458355a0418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:52.762934 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def8a8a1-77f0-41a3-b0a9-3458355a0418 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:52.763240 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def8a8a1-77f0-41a3-b0a9-3458355a0418 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:52.763508 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def8a8a1-77f0-41a3-b0a9-3458355a0418 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:52.769097 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 295/589] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:29:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:52.769097 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2829e941-b047-4aad-8622-8f98af1d6fbd None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:52.775580 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def8a8a1-77f0-41a3-b0a9-3458355a0418 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:52.777441 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def8a8a1-77f0-41a3-b0a9-3458355a0418 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:52.789689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-def8a8a1-77f0-41a3-b0a9-3458355a0418 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:29:52.789689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:52.789689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:52.789689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:52.789689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:52.789689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:52.789689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:52.789689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:52.789689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:52.789689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:52.789689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:52.789689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:52.789689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:29:52.789689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:29:52.789689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:29:52.790575 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:29:52.790575 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:52.790575 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:52.790575 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:52.790575 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:52.790575 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:52.790575 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:52.790575 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:52.790575 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:52.790575 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:52.790575 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:52.790575 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:52.790575 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:52.790575 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:52.790575 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:52.790575 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:52.790575 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:52.791533 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:29:52.791533 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:29:52.791533 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:29:52.791533 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:29:52.791533 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:29:52.791533 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:29:52.791533 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:29:52.791533 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:52.791533 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 295/590] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:29:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:52.792048 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def8a8a1-77f0-41a3-b0a9-3458355a0418 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:52.804137 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:52.879131 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:52.880084 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2829e941-b047-4aad-8622-8f98af1d6fbd None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:52.905874 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def8a8a1-77f0-41a3-b0a9-3458355a0418 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:52.918984 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ee87be-94b0-4b83-82c9-a86c514ddb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:52.919617 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee87be-94b0-4b83-82c9-a86c514ddb2c None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:52.919891 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee87be-94b0-4b83-82c9-a86c514ddb2c None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:52.920151 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee87be-94b0-4b83-82c9-a86c514ddb2c None admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:52.920840 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ee87be-94b0-4b83-82c9-a86c514ddb2c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:52.921140 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ee87be-94b0-4b83-82c9-a86c514ddb2c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:52.922565 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ee87be-94b0-4b83-82c9-a86c514ddb2c None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:52.938835 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 296/591] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 30 14:29:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:52.938835 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ee87be-94b0-4b83-82c9-a86c514ddb2c None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:52.946460 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10bf225-066a-42fb-9732-716a5686e2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:52.947063 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10bf225-066a-42fb-9732-716a5686e2a9 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:52.947478 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10bf225-066a-42fb-9732-716a5686e2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:52.947754 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10bf225-066a-42fb-9732-716a5686e2a9 None admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:52.949011 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10bf225-066a-42fb-9732-716a5686e2a9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:52.949310 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10bf225-066a-42fb-9732-716a5686e2a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:52.950876 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10bf225-066a-42fb-9732-716a5686e2a9 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:52.974135 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 296/592] 10.222.0.30 () {56 vars in 1139 bytes} [Tue Aug 30 14:29:52 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:52.974135 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10bf225-066a-42fb-9732-716a5686e2a9 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:53.029746 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:53.042851 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ee87be-94b0-4b83-82c9-a86c514ddb2c None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:53.079760 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd17872-325a-4168-9df9-8585d2b01817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:53.080397 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd17872-325a-4168-9df9-8585d2b01817 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:53.080704 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd17872-325a-4168-9df9-8585d2b01817 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:53.080950 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd17872-325a-4168-9df9-8585d2b01817 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:53.081175 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10bf225-066a-42fb-9732-716a5686e2a9 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:53.088607 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd17872-325a-4168-9df9-8585d2b01817 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:53.090847 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd17872-325a-4168-9df9-8585d2b01817 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:53.102127 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-7fd17872-325a-4168-9df9-8585d2b01817 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 14:29:53.102127 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:53.102127 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:53.102127 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:53.102127 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:53.102127 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:53.102127 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:53.102127 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:53.102127 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:53.102127 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:53.102127 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:53.102127 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:53.102127 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:29:53.102127 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:29:53.102127 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:29:53.102127 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:29:53.103074 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:53.103074 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:53.103074 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:29:53.103074 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:29:53.103074 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:29:53.103074 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:29:53.103074 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:53.103074 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:53.103074 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:53.103074 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:53.103074 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:53.103074 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:53.103074 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:53.103074 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:53.103074 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:53.103074 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:53.103074 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:53.103074 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:53.104367 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:53.104367 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:53.104367 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:29:53.104367 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:29:53.104367 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:29:53.104367 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:29:53.104367 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:29:53.104367 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:29:53.104367 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 14:29:53.104367 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:53.104367 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 297/593] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:29:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:53.104367 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd17872-325a-4168-9df9-8585d2b01817 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:53.117228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:53.123312 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0075656-77a8-4425-9438-b020dea2b0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:53.123939 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0075656-77a8-4425-9438-b020dea2b0f2 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:53.124212 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0075656-77a8-4425-9438-b020dea2b0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:53.124487 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0075656-77a8-4425-9438-b020dea2b0f2 None admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:53.125183 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0075656-77a8-4425-9438-b020dea2b0f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:53.125486 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0075656-77a8-4425-9438-b020dea2b0f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:53.126908 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0075656-77a8-4425-9438-b020dea2b0f2 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:53.144837 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 297/594] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 30 14:29:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:53.144837 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0075656-77a8-4425-9438-b020dea2b0f2 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:53.221062 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd17872-325a-4168-9df9-8585d2b01817 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:53.257188 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f123e-6809-4a07-9e05-b16539f1fff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:53.257769 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f123e-6809-4a07-9e05-b16539f1fff8 None admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:53.258049 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f123e-6809-4a07-9e05-b16539f1fff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:53.258330 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f123e-6809-4a07-9e05-b16539f1fff8 None admin] PATH_INFO: `/v3/projects/01cf71ca5abf4576b89d10c85af6d2de/users/8878de66a03044e38e507028853a0cd1/roles/1316db16eac54629a283545f1601a236` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:53.261458 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0075656-77a8-4425-9438-b020dea2b0f2 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:53.273961 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90f123e-6809-4a07-9e05-b16539f1fff8 None admin] RBAC: Authorizing `identity:create_grant(project_id=01cf71ca5abf4576b89d10c85af6d2de, user_id=8878de66a03044e38e507028853a0cd1, role_id=1316db16eac54629a283545f1601a236)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:53.275428 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90f123e-6809-4a07-9e05-b16539f1fff8 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:53.284509 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:53.299240 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 298/595] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 30 14:29:53 2022] PUT /identity/v3/projects/01cf71ca5abf4576b89d10c85af6d2de/users/8878de66a03044e38e507028853a0cd1/roles/1316db16eac54629a283545f1601a236 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:29:53.299240 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f123e-6809-4a07-9e05-b16539f1fff8 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:53.320754 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eaf733-ca49-4305-9ddf-62e2b62558d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:53.321859 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eaf733-ca49-4305-9ddf-62e2b62558d9 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:53.322632 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eaf733-ca49-4305-9ddf-62e2b62558d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:53.322912 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eaf733-ca49-4305-9ddf-62e2b62558d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:53.342170 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eaf733-ca49-4305-9ddf-62e2b62558d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:53.344220 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eaf733-ca49-4305-9ddf-62e2b62558d9 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:53.363893 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-83eaf733-ca49-4305-9ddf-62e2b62558d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:53.363893 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:53.363893 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:53.363893 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:53.363893 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:53.363893 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:53.363893 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:53.363893 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:53.363893 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:53.363893 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:53.363893 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:53.363893 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:53.363893 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:29:53.363893 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:29:53.363893 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:29:53.363893 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:29:53.364914 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:53.364914 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:53.364914 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:53.364914 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:53.364914 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:53.364914 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:53.364914 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:53.364914 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:53.364914 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:53.364914 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:53.364914 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:53.364914 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:53.364914 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:53.364914 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:53.364914 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:53.364914 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:53.364914 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:29:53.365796 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:29:53.365796 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:53.365796 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:53.370004 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 298/596] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:29:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:53.371553 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eaf733-ca49-4305-9ddf-62e2b62558d9 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:53.415131 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f123e-6809-4a07-9e05-b16539f1fff8 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:53.429451 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:53.457884 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807b1a06-8c1f-4536-bffe-15e392ef1d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:53.458573 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b1a06-8c1f-4536-bffe-15e392ef1d3b None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:53.458878 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b1a06-8c1f-4536-bffe-15e392ef1d3b None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:53.459150 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b1a06-8c1f-4536-bffe-15e392ef1d3b None admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:53.459911 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807b1a06-8c1f-4536-bffe-15e392ef1d3b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:53.460213 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807b1a06-8c1f-4536-bffe-15e392ef1d3b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:53.461650 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807b1a06-8c1f-4536-bffe-15e392ef1d3b None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:53.481812 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 299/597] 10.222.0.30 () {56 vars in 1137 bytes} [Tue Aug 30 14:29:53 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:53.483299 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807b1a06-8c1f-4536-bffe-15e392ef1d3b None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:53.499756 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eaf733-ca49-4305-9ddf-62e2b62558d9 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:53.545535 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:53.549050 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641399aa-0464-4591-9ed9-249f9bd67f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:53.551037 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641399aa-0464-4591-9ed9-249f9bd67f6e None admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:53.551750 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641399aa-0464-4591-9ed9-249f9bd67f6e None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:53.552200 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641399aa-0464-4591-9ed9-249f9bd67f6e None admin] PATH_INFO: `/v3/projects/01cf71ca5abf4576b89d10c85af6d2de/users/6889402e54424bbd8d42c7bb759cf700/roles/1316db16eac54629a283545f1601a236` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:53.574160 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641399aa-0464-4591-9ed9-249f9bd67f6e None admin] RBAC: Authorizing `identity:create_grant(project_id=01cf71ca5abf4576b89d10c85af6d2de, user_id=6889402e54424bbd8d42c7bb759cf700, role_id=1316db16eac54629a283545f1601a236)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:53.576242 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641399aa-0464-4591-9ed9-249f9bd67f6e None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:53.594887 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 299/598] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 30 14:29:53 2022] PUT /identity/v3/projects/01cf71ca5abf4576b89d10c85af6d2de/users/6889402e54424bbd8d42c7bb759cf700/roles/1316db16eac54629a283545f1601a236 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:29:53.596665 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641399aa-0464-4591-9ed9-249f9bd67f6e None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:53.598570 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807b1a06-8c1f-4536-bffe-15e392ef1d3b None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:53.662955 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0fee6-cbab-424c-911b-9c13d8d7f501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:53.664165 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0fee6-cbab-424c-911b-9c13d8d7f501 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:53.664997 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0fee6-cbab-424c-911b-9c13d8d7f501 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:53.665918 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0fee6-cbab-424c-911b-9c13d8d7f501 None admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:53.666609 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf0fee6-cbab-424c-911b-9c13d8d7f501 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:53.666922 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf0fee6-cbab-424c-911b-9c13d8d7f501 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:53.669167 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf0fee6-cbab-424c-911b-9c13d8d7f501 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:53.684248 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 300/599] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:29:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:53.686482 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c114d-a1d9-4b9e-a5ef-0761b4c09d1f None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:53.686777 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c114d-a1d9-4b9e-a5ef-0761b4c09d1f None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:53.687057 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c114d-a1d9-4b9e-a5ef-0761b4c09d1f None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:53.693678 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 301/600] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:53.693678 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953c114d-a1d9-4b9e-a5ef-0761b4c09d1f None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:53.731677 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:53.740815 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641399aa-0464-4591-9ed9-249f9bd67f6e None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:53.786979 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0ea63f-c1eb-4987-a8f8-9302ebe1f175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:53.786979 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ea63f-c1eb-4987-a8f8-9302ebe1f175 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:53.787279 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ea63f-c1eb-4987-a8f8-9302ebe1f175 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:53.787545 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ea63f-c1eb-4987-a8f8-9302ebe1f175 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:53.799194 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0ea63f-c1eb-4987-a8f8-9302ebe1f175 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:53.800620 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0ea63f-c1eb-4987-a8f8-9302ebe1f175 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:53.821063 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-5a0ea63f-c1eb-4987-a8f8-9302ebe1f175 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:53.821063 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:53.821063 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:53.821063 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:53.821063 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:53.821063 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:53.821063 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:53.821063 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:53.821063 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:53.821063 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:53.821063 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:53.821063 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:53.821063 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:29:53.821063 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:29:53.821063 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:29:53.821063 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:29:53.822070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:53.822070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:53.822070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:53.822070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:53.822070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:53.822070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:53.822070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:53.822070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:53.822070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:53.822070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:53.822070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:53.822070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:53.822070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:53.822070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:53.822070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:53.822070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:53.822070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:29:53.822967 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:29:53.822967 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:53.822967 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:53.826835 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 300/601] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:29:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:53.828444 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1411a-a04c-4162-a9d6-3bff0ff02289 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:53.830186 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1411a-a04c-4162-a9d6-3bff0ff02289 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:53.830467 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1411a-a04c-4162-a9d6-3bff0ff02289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:53.843554 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953c114d-a1d9-4b9e-a5ef-0761b4c09d1f None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:53.850032 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-17a1411a-a04c-4162-a9d6-3bff0ff02289 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:53.851355 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-17a1411a-a04c-4162-a9d6-3bff0ff02289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:53.862879 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:53.883528 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1181c9d0-40a4-4da7-ac35-42ccf7ad97da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:53.884512 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181c9d0-40a4-4da7-ac35-42ccf7ad97da None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:53.884997 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181c9d0-40a4-4da7-ac35-42ccf7ad97da None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:53.885527 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1181c9d0-40a4-4da7-ac35-42ccf7ad97da None admin] PATH_INFO: `/v3/projects/service` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:53.893117 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1181c9d0-40a4-4da7-ac35-42ccf7ad97da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:53.894875 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1181c9d0-40a4-4da7-ac35-42ccf7ad97da None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:53.907117 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-1181c9d0-40a4-4da7-ac35-42ccf7ad97da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:29:53.907117 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:53.907117 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:53.907117 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:53.907117 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:53.907117 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:53.907117 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:53.907117 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:53.907117 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:53.907117 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:53.907117 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:53.907117 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:53.907117 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:29:53.907117 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:29:53.907117 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:29:53.908089 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:29:53.908089 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:53.908089 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:53.908089 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:53.908089 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:53.908089 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:53.908089 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:53.908089 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:53.908089 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:53.908089 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:53.908089 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:53.908089 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:53.908089 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:53.908089 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:53.908089 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:53.908089 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:53.908089 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:53.908982 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:29:53.908982 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:29:53.908982 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:29:53.908982 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:29:53.908982 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:29:53.908982 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:29:53.908982 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:29:53.908982 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:53.911430 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 302/602] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:29:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:53.912558 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1181c9d0-40a4-4da7-ac35-42ccf7ad97da None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:53.934865 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 301/603] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:53.936991 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a1411a-a04c-4162-a9d6-3bff0ff02289 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:54.088256 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:54.098117 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1181c9d0-40a4-4da7-ac35-42ccf7ad97da None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:54.141470 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a1411a-a04c-4162-a9d6-3bff0ff02289 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:54.147469 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d0dbc1-08b2-4e3e-923b-1a42e9c5a932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:54.148717 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d0dbc1-08b2-4e3e-923b-1a42e9c5a932 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:54.148717 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d0dbc1-08b2-4e3e-923b-1a42e9c5a932 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:54.148717 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d0dbc1-08b2-4e3e-923b-1a42e9c5a932 None admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:54.149251 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d0dbc1-08b2-4e3e-923b-1a42e9c5a932 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:54.149556 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d0dbc1-08b2-4e3e-923b-1a42e9c5a932 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:54.150932 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d0dbc1-08b2-4e3e-923b-1a42e9c5a932 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:54.171290 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 303/604] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:29:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:54.172612 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d0dbc1-08b2-4e3e-923b-1a42e9c5a932 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:54.210692 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:54.240839 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58a0d35-ffca-42ad-a8d4-a6f50aff4d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:54.241550 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a0d35-ffca-42ad-a8d4-a6f50aff4d70 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:54.241981 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a0d35-ffca-42ad-a8d4-a6f50aff4d70 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:54.242315 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a0d35-ffca-42ad-a8d4-a6f50aff4d70 None admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:54.243184 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58a0d35-ffca-42ad-a8d4-a6f50aff4d70 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:54.243626 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58a0d35-ffca-42ad-a8d4-a6f50aff4d70 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:54.247230 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58a0d35-ffca-42ad-a8d4-a6f50aff4d70 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:54.271237 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 302/605] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 30 14:29:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:54.273682 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58a0d35-ffca-42ad-a8d4-a6f50aff4d70 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:54.291236 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d0dbc1-08b2-4e3e-923b-1a42e9c5a932 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:54.327084 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c93402-4eca-45f3-9894-8e6f8d0926c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:54.331241 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c93402-4eca-45f3-9894-8e6f8d0926c4 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:54.331241 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c93402-4eca-45f3-9894-8e6f8d0926c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:54.331241 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c93402-4eca-45f3-9894-8e6f8d0926c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:54.343304 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c93402-4eca-45f3-9894-8e6f8d0926c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:54.343304 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c93402-4eca-45f3-9894-8e6f8d0926c4 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:54.363239 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-a3c93402-4eca-45f3-9894-8e6f8d0926c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:29:54.363239 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:54.363239 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:54.363239 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:54.363239 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:54.363239 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:54.363239 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:54.363239 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:54.363239 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:54.363239 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:54.363239 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:54.363239 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:54.363239 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:29:54.363239 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:29:54.363239 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:29:54.364162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:29:54.364162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:54.364162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:54.364162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:54.364162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:54.364162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:54.364162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:54.364162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:54.364162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:54.364162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:54.364162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:54.364162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:54.364162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:54.364162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:54.364162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:54.364162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:54.364162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:54.365040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:29:54.365040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:29:54.365040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:29:54.365040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:29:54.365040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:29:54.365040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:29:54.365040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:29:54.365040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:54.371234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 304/606] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:29:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:29:54.371234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c93402-4eca-45f3-9894-8e6f8d0926c4 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:54.466124 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58a0d35-ffca-42ad-a8d4-a6f50aff4d70 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:54.477609 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:54.484506 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c93402-4eca-45f3-9894-8e6f8d0926c4 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:54.519900 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b9d724-f07b-4011-8939-815c994bb865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:54.520566 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9d724-f07b-4011-8939-815c994bb865 None admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:54.520844 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9d724-f07b-4011-8939-815c994bb865 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:54.521117 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9d724-f07b-4011-8939-815c994bb865 None admin] PATH_INFO: `/v3/projects/01cf71ca5abf4576b89d10c85af6d2de/users/9e14e86e085d4987aece69fe64867ce6/roles/1316db16eac54629a283545f1601a236` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:54.539980 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d1bbe-7511-471f-9eeb-a37b43547993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:54.541039 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d1bbe-7511-471f-9eeb-a37b43547993 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:54.541606 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d1bbe-7511-471f-9eeb-a37b43547993 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:54.542695 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d1bbe-7511-471f-9eeb-a37b43547993 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:54.543434 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b9d724-f07b-4011-8939-815c994bb865 None admin] RBAC: Authorizing `identity:create_grant(project_id=01cf71ca5abf4576b89d10c85af6d2de, user_id=9e14e86e085d4987aece69fe64867ce6, role_id=1316db16eac54629a283545f1601a236)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:54.545154 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b9d724-f07b-4011-8939-815c994bb865 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:54.549378 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:54.568061 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9d1bbe-7511-471f-9eeb-a37b43547993 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:54.575253 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 305/607] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 30 14:29:54 2022] PUT /identity/v3/projects/01cf71ca5abf4576b89d10c85af6d2de/users/9e14e86e085d4987aece69fe64867ce6/roles/1316db16eac54629a283545f1601a236 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:29:54.575253 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b9d724-f07b-4011-8939-815c994bb865 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:54.579311 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-8e9d1bbe-7511-471f-9eeb-a37b43547993 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:29:54.579311 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:54.579311 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:54.579311 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:54.579311 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:54.579311 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:54.579311 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:54.579311 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:54.579311 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:54.579311 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:54.579311 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:54.579311 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:54.579311 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:54.579311 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:54.579311 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:54.579311 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:54.580479 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:54.580479 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:54.580479 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:54.580479 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:54.580479 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:54.580479 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:54.580479 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:54.580479 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:54.580479 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:54.580479 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:54.580479 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:54.580479 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:54.580479 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:54.580479 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:54.580479 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:54.580479 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:54.581479 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:54.581479 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:54.581479 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:54.581479 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:54.581479 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:54.581479 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:54.581479 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:54.581479 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:54.581479 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:54.581479 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:54.581479 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:29:54.581479 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:54.592035 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 303/608] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 30 14:29:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:54.600670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d1bbe-7511-471f-9eeb-a37b43547993 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:54.701725 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b9d724-f07b-4011-8939-815c994bb865 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:54.740581 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d9f79-06ca-4121-a00c-f30c1a090b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:54.741489 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d9f79-06ca-4121-a00c-f30c1a090b1d None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:54.741921 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d9f79-06ca-4121-a00c-f30c1a090b1d None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:54.742342 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d9f79-06ca-4121-a00c-f30c1a090b1d None admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:54.743202 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79d9f79-06ca-4121-a00c-f30c1a090b1d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:54.743685 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79d9f79-06ca-4121-a00c-f30c1a090b1d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:54.746156 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79d9f79-06ca-4121-a00c-f30c1a090b1d None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:54.764283 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 306/609] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 30 14:29:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:54.765931 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d9f79-06ca-4121-a00c-f30c1a090b1d None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:54.775922 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:54.900853 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d9f79-06ca-4121-a00c-f30c1a090b1d None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:54.906816 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d1bbe-7511-471f-9eeb-a37b43547993 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:54.938566 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0144de9-b969-411d-9112-eb3d897ddbf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:54.939514 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0144de9-b969-411d-9112-eb3d897ddbf4 None admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:54.939944 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0144de9-b969-411d-9112-eb3d897ddbf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:54.940368 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0144de9-b969-411d-9112-eb3d897ddbf4 None admin] PATH_INFO: `/v3/projects/01cf71ca5abf4576b89d10c85af6d2de/users/6fda7075198d48da86aa9c0dab6ff7ca/roles/1316db16eac54629a283545f1601a236` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:54.957398 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0144de9-b969-411d-9112-eb3d897ddbf4 None admin] RBAC: Authorizing `identity:create_grant(project_id=01cf71ca5abf4576b89d10c85af6d2de, user_id=6fda7075198d48da86aa9c0dab6ff7ca, role_id=1316db16eac54629a283545f1601a236)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:54.959094 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0144de9-b969-411d-9112-eb3d897ddbf4 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:54.987402 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 307/610] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 30 14:29:54 2022] PUT /identity/v3/projects/01cf71ca5abf4576b89d10c85af6d2de/users/6fda7075198d48da86aa9c0dab6ff7ca/roles/1316db16eac54629a283545f1601a236 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:29:55.016847 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d8f6b6-61d7-493c-95ec-a159a987e332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:55.017672 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d8f6b6-61d7-493c-95ec-a159a987e332 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:55.023671 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d8f6b6-61d7-493c-95ec-a159a987e332 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:55.024100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d8f6b6-61d7-493c-95ec-a159a987e332 None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:55.024952 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d8f6b6-61d7-493c-95ec-a159a987e332 None admin] RBAC: Adding query filter params (name=service) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:55.025365 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d8f6b6-61d7-493c-95ec-a159a987e332 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:55.027469 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d8f6b6-61d7-493c-95ec-a159a987e332 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:55.065214 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 304/611] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 30 14:29:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 465 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:29:55.070712 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0144de9-b969-411d-9112-eb3d897ddbf4 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:55.189032 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:55.256462 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0144de9-b969-411d-9112-eb3d897ddbf4 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:55.302680 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cadd343-f1cc-4ae0-b2c2-73e7a57fbdd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:55.303727 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cadd343-f1cc-4ae0-b2c2-73e7a57fbdd0 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:55.304157 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cadd343-f1cc-4ae0-b2c2-73e7a57fbdd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:55.304575 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cadd343-f1cc-4ae0-b2c2-73e7a57fbdd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:55.316582 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cadd343-f1cc-4ae0-b2c2-73e7a57fbdd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:55.318261 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cadd343-f1cc-4ae0-b2c2-73e7a57fbdd0 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:55.335277 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-2cadd343-f1cc-4ae0-b2c2-73e7a57fbdd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:55.335277 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:55.335277 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:55.335277 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:55.335277 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:55.335277 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:55.335277 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:55.335277 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:55.335277 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:55.335277 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:55.335277 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:55.335277 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:55.335277 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:29:55.335277 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:29:55.335277 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:29:55.335277 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:29:55.336462 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:55.336462 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:55.336462 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:55.336462 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:55.336462 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:55.336462 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:55.336462 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:55.336462 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:55.336462 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:55.336462 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:55.336462 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:55.336462 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:55.336462 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:55.336462 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:55.336462 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:55.336462 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:55.336462 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:29:55.337563 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:29:55.337563 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:55.337563 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:55.340127 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 308/612] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:29:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:55.345217 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d8f6b6-61d7-493c-95ec-a159a987e332 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:55.431980 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:55.495053 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d8f6b6-61d7-493c-95ec-a159a987e332 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:55.559262 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f51a191-4f51-4ca7-8ecc-41ccc4538948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:55.559894 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51a191-4f51-4ca7-8ecc-41ccc4538948 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:55.560165 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51a191-4f51-4ca7-8ecc-41ccc4538948 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:55.560468 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51a191-4f51-4ca7-8ecc-41ccc4538948 None admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:55.561196 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f51a191-4f51-4ca7-8ecc-41ccc4538948 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:55.561496 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f51a191-4f51-4ca7-8ecc-41ccc4538948 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:55.562953 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f51a191-4f51-4ca7-8ecc-41ccc4538948 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:55.565790 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e759e9-889e-480a-a268-0dff0bd42557 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:55.566079 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e759e9-889e-480a-a268-0dff0bd42557 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:55.566354 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e759e9-889e-480a-a268-0dff0bd42557 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:55.568021 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 309/613] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:55.577494 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 305/614] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:29:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:55.581966 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf63e9-face-4a99-827d-3d3adcdb7528 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:55.581966 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf63e9-face-4a99-827d-3d3adcdb7528 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:55.581966 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf63e9-face-4a99-827d-3d3adcdb7528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:55.583661 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f51a191-4f51-4ca7-8ecc-41ccc4538948 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:55.624338 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-cfcf63e9-face-4a99-827d-3d3adcdb7528 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:55.625576 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-cfcf63e9-face-4a99-827d-3d3adcdb7528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:55.712098 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f51a191-4f51-4ca7-8ecc-41ccc4538948 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:55.733948 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:55.755471 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ee288-5836-450c-9f83-e8efda1c4a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:55.756029 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ee288-5836-450c-9f83-e8efda1c4a38 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:55.756339 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ee288-5836-450c-9f83-e8efda1c4a38 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:55.756579 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ee288-5836-450c-9f83-e8efda1c4a38 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:55.766183 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8ee288-5836-450c-9f83-e8efda1c4a38 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:55.767702 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8ee288-5836-450c-9f83-e8efda1c4a38 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:55.781927 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-df8ee288-5836-450c-9f83-e8efda1c4a38 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 14:29:55.781927 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:55.781927 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:55.781927 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:55.781927 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:55.781927 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:55.781927 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:55.781927 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:55.781927 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:55.781927 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:55.781927 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:55.781927 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:55.781927 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:29:55.781927 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:29:55.781927 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:29:55.781927 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:29:55.782910 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:55.782910 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:55.782910 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:29:55.782910 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:29:55.782910 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:29:55.782910 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:29:55.782910 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:55.782910 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:55.782910 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:55.782910 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:55.782910 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:55.782910 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:55.782910 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:55.782910 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:55.782910 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:55.782910 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:55.782910 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:55.782910 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:55.783883 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:55.783883 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:55.783883 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:29:55.783883 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:29:55.783883 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:29:55.783883 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:29:55.783883 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:29:55.783883 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:29:55.783883 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 14:29:55.783883 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:55.783883 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 306/615] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 30 14:29:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:55.790727 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ee288-5836-450c-9f83-e8efda1c4a38 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:55.836947 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 310/616] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:55.975566 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ee288-5836-450c-9f83-e8efda1c4a38 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:56.027499 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261be57f-864e-4364-a0a8-b453cd6736db None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:56.027918 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261be57f-864e-4364-a0a8-b453cd6736db None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:56.028494 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261be57f-864e-4364-a0a8-b453cd6736db None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:56.030733 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 311/617] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:56.037484 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a86b66-8af9-4b4f-972f-18840fee7036 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:56.037933 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a86b66-8af9-4b4f-972f-18840fee7036 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:56.038396 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a86b66-8af9-4b4f-972f-18840fee7036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:56.038986 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73089e30-0c5c-4de7-9614-e5620017b414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:56.039872 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73089e30-0c5c-4de7-9614-e5620017b414 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:56.040638 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73089e30-0c5c-4de7-9614-e5620017b414 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:56.041543 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73089e30-0c5c-4de7-9614-e5620017b414 None admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:56.043071 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73089e30-0c5c-4de7-9614-e5620017b414 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:56.043634 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73089e30-0c5c-4de7-9614-e5620017b414 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:56.045273 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73089e30-0c5c-4de7-9614-e5620017b414 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:56.068590 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 307/618] 10.222.0.30 () {56 vars in 1135 bytes} [Tue Aug 30 14:29:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:56.074763 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73089e30-0c5c-4de7-9614-e5620017b414 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:56.088145 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-07a86b66-8af9-4b4f-972f-18840fee7036 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:56.089771 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-07a86b66-8af9-4b4f-972f-18840fee7036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:56.141200 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:56.148459 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:56.196644 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 312/619] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:29:56.252817 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a86b66-8af9-4b4f-972f-18840fee7036 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:56.300397 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73089e30-0c5c-4de7-9614-e5620017b414 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:56.366344 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ceea64-229e-44b6-8832-b0c2bf23b641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:56.367248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ceea64-229e-44b6-8832-b0c2bf23b641 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:56.367674 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ceea64-229e-44b6-8832-b0c2bf23b641 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:56.368093 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ceea64-229e-44b6-8832-b0c2bf23b641 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:56.371272 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a86b66-8af9-4b4f-972f-18840fee7036 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:56.389630 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ceea64-229e-44b6-8832-b0c2bf23b641 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:56.391353 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ceea64-229e-44b6-8832-b0c2bf23b641 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:56.414914 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-38ceea64-229e-44b6-8832-b0c2bf23b641 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:56.414914 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:56.414914 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:56.414914 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:56.414914 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:56.414914 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:56.414914 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:56.414914 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:56.414914 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:56.414914 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:56.414914 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:56.414914 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:56.414914 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:29:56.414914 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:29:56.414914 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:29:56.414914 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:29:56.416176 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:56.416176 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:56.416176 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:56.416176 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:56.416176 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:56.416176 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:56.416176 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:56.416176 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:56.416176 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:56.416176 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:56.416176 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:56.416176 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:56.416176 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:56.416176 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:56.416176 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:56.416176 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:56.416176 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:29:56.418007 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:29:56.418007 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:56.418007 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:56.420856 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 308/620] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:29:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:29:56.428008 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ceea64-229e-44b6-8832-b0c2bf23b641 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:56.446231 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f87217-4ec4-40b9-b550-0585b3bba3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:56.447100 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f87217-4ec4-40b9-b550-0585b3bba3a3 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:56.447606 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f87217-4ec4-40b9-b550-0585b3bba3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:56.448023 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f87217-4ec4-40b9-b550-0585b3bba3a3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:56.469195 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f87217-4ec4-40b9-b550-0585b3bba3a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:56.474318 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-82f87217-4ec4-40b9-b550-0585b3bba3a3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:29:56.474318 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:56.474318 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:56.474318 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:56.474318 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:56.474318 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:56.474318 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:56.474318 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:56.474318 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:56.474318 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:56.474318 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:56.474318 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:56.474318 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:56.474318 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:56.474318 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:56.474318 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:56.475604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:56.475604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:56.475604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:56.475604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:56.475604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:56.475604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:56.475604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:56.475604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:56.475604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:56.475604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:56.475604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:56.475604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:56.475604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:56.475604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:56.475604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:56.475604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:56.476638 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:56.476638 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:56.476638 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:56.476638 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:56.476638 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:56.476638 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:56.476638 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:56.476638 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:56.476638 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:56.476638 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:56.476638 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:29:56.476638 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:56.478844 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 313/621] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 30 14:29:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:29:56.488298 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f87217-4ec4-40b9-b550-0585b3bba3a3 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:56.526430 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:56.574263 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:56.591885 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ceea64-229e-44b6-8832-b0c2bf23b641 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:56.606127 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f87217-4ec4-40b9-b550-0585b3bba3a3 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:56.630411 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ea048-4367-44e0-a384-b05f8be4a436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:56.631168 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ea048-4367-44e0-a384-b05f8be4a436 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:56.631494 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ea048-4367-44e0-a384-b05f8be4a436 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:56.631840 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ea048-4367-44e0-a384-b05f8be4a436 None admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:56.632570 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465ea048-4367-44e0-a384-b05f8be4a436 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:56.633049 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465ea048-4367-44e0-a384-b05f8be4a436 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:56.634586 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465ea048-4367-44e0-a384-b05f8be4a436 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:56.646029 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa153ee9-99e0-4d7a-8d8c-ae07dd1bedcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:56.646636 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa153ee9-99e0-4d7a-8d8c-ae07dd1bedcf None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:56.646914 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa153ee9-99e0-4d7a-8d8c-ae07dd1bedcf None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:56.647183 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa153ee9-99e0-4d7a-8d8c-ae07dd1bedcf None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:56.647885 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa153ee9-99e0-4d7a-8d8c-ae07dd1bedcf None admin] RBAC: Adding query filter params (name=service) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:56.648187 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa153ee9-99e0-4d7a-8d8c-ae07dd1bedcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:56.649565 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa153ee9-99e0-4d7a-8d8c-ae07dd1bedcf None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:56.649748 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 309/622] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:29:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:56.650858 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ea048-4367-44e0-a384-b05f8be4a436 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:56.658607 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 314/623] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 30 14:29:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:56.659823 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa153ee9-99e0-4d7a-8d8c-ae07dd1bedcf None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:56.758801 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:56.765367 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ea048-4367-44e0-a384-b05f8be4a436 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:56.769124 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa153ee9-99e0-4d7a-8d8c-ae07dd1bedcf None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:56.803595 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec5e240-f361-415c-b6d6-14a16dc273e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:56.804667 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec5e240-f361-415c-b6d6-14a16dc273e4 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:56.805094 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec5e240-f361-415c-b6d6-14a16dc273e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:56.805451 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73f0357-b9e1-4f14-8e10-cc8809545b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:56.805700 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec5e240-f361-415c-b6d6-14a16dc273e4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:56.806083 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f0357-b9e1-4f14-8e10-cc8809545b6d None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:56.806363 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f0357-b9e1-4f14-8e10-cc8809545b6d None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:56.806638 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f0357-b9e1-4f14-8e10-cc8809545b6d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:56.815001 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73f0357-b9e1-4f14-8e10-cc8809545b6d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:56.816581 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73f0357-b9e1-4f14-8e10-cc8809545b6d None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:56.818756 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec5e240-f361-415c-b6d6-14a16dc273e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:56.823951 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-6ec5e240-f361-415c-b6d6-14a16dc273e4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:29:56.823951 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:56.823951 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:56.823951 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:56.823951 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:56.823951 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:56.823951 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:56.823951 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:56.823951 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:56.823951 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:56.823951 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:56.823951 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:56.823951 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:56.823951 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:56.823951 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:56.823951 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:56.825122 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:56.825122 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:56.825122 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:56.825122 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:56.825122 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:56.825122 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:56.825122 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:56.825122 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:56.825122 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:56.825122 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:56.825122 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:56.825122 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:56.825122 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:56.825122 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:56.825122 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:56.825122 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:56.826159 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:56.826159 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:56.826159 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:56.826159 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:56.826159 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:56.826159 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:56.826159 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:56.826159 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:56.826159 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:56.826159 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:56.826159 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:29:56.826159 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:56.828195 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-d73f0357-b9e1-4f14-8e10-cc8809545b6d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:29:56.828195 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:56.828195 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:56.828195 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:56.828195 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:56.828195 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:56.828195 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:56.828195 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:56.828195 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:56.828195 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:56.828195 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:56.828195 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:56.828195 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:29:56.828195 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:29:56.828195 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:29:56.829233 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:29:56.829233 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:56.829233 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:56.829233 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:56.829233 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:56.829233 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:56.829233 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:56.829233 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:56.829233 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:56.829233 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:56.829233 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:56.829233 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:56.829233 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:56.829233 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:56.829233 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:56.829233 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:56.829233 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:56.830439 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:29:56.830439 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:29:56.830439 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:29:56.830439 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:29:56.830439 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:29:56.830439 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:29:56.830439 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:29:56.830439 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:56.830439 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 315/624] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:29:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:56.830439 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73f0357-b9e1-4f14-8e10-cc8809545b6d None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:56.831502 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 310/625] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 30 14:29:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:56.838183 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec5e240-f361-415c-b6d6-14a16dc273e4 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:56.846756 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:56.940227 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73f0357-b9e1-4f14-8e10-cc8809545b6d None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:56.941549 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec5e240-f361-415c-b6d6-14a16dc273e4 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:56.978105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b084685-b80b-4655-a8a2-9806f2e7251f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:56.979542 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b084685-b80b-4655-a8a2-9806f2e7251f None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:56.979723 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab60fb-3773-424e-a4d4-daff8652f0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:56.980321 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab60fb-3773-424e-a4d4-daff8652f0e6 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:56.980527 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab60fb-3773-424e-a4d4-daff8652f0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:56.980716 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab60fb-3773-424e-a4d4-daff8652f0e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:56.981185 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b084685-b80b-4655-a8a2-9806f2e7251f None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:56.981385 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab60fb-3773-424e-a4d4-daff8652f0e6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:56.981690 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab60fb-3773-424e-a4d4-daff8652f0e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:56.982057 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b084685-b80b-4655-a8a2-9806f2e7251f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:56.983265 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab60fb-3773-424e-a4d4-daff8652f0e6 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:56.992795 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 311/626] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 30 14:29:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:56.993546 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab60fb-3773-424e-a4d4-daff8652f0e6 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:56.995364 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b084685-b80b-4655-a8a2-9806f2e7251f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:56.997236 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b084685-b80b-4655-a8a2-9806f2e7251f None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:57.013929 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-6b084685-b80b-4655-a8a2-9806f2e7251f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:57.013929 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:57.013929 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:57.013929 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:57.013929 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:57.013929 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:57.013929 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:57.013929 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:57.013929 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:57.013929 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:57.013929 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:57.013929 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:57.013929 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:29:57.013929 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:29:57.013929 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:29:57.013929 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:29:57.014938 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:57.014938 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:57.014938 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:57.014938 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:57.014938 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:57.014938 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:57.014938 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:57.014938 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:57.014938 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:57.014938 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:57.014938 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:57.014938 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:57.014938 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:57.014938 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:57.014938 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:57.014938 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:57.014938 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:29:57.015875 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:29:57.015875 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:57.015875 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:57.018035 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 316/627] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:29:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:57.019021 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b084685-b80b-4655-a8a2-9806f2e7251f None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:57.022328 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:57.071918 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:57.107733 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab60fb-3773-424e-a4d4-daff8652f0e6 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:57.134015 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b084685-b80b-4655-a8a2-9806f2e7251f None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:57.144778 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e95851-d791-434d-86e2-e50da580562a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:57.145646 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e95851-d791-434d-86e2-e50da580562a None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:57.145781 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e95851-d791-434d-86e2-e50da580562a None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:57.146032 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e95851-d791-434d-86e2-e50da580562a None admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:57.146736 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e95851-d791-434d-86e2-e50da580562a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:57.147141 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e95851-d791-434d-86e2-e50da580562a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:57.148504 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e95851-d791-434d-86e2-e50da580562a None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:57.162325 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 312/628] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 30 14:29:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:57.163407 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e95851-d791-434d-86e2-e50da580562a None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:57.174594 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a294865-8c70-46bc-b243-53f337cc0e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:57.175253 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a294865-8c70-46bc-b243-53f337cc0e8b None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:57.175536 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a294865-8c70-46bc-b243-53f337cc0e8b None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:57.175808 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a294865-8c70-46bc-b243-53f337cc0e8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:57.186933 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a294865-8c70-46bc-b243-53f337cc0e8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:57.188434 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a294865-8c70-46bc-b243-53f337cc0e8b None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:57.202687 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-9a294865-8c70-46bc-b243-53f337cc0e8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:57.202687 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:57.202687 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:57.202687 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:57.202687 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:57.202687 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:57.202687 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:57.202687 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:57.202687 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:57.202687 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:57.202687 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:57.202687 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:57.202687 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:29:57.202687 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:29:57.202687 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:29:57.202687 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:29:57.203711 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:57.203711 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:57.203711 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:57.203711 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:57.203711 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:57.203711 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:57.203711 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:57.203711 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:57.203711 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:57.203711 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:57.203711 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:57.203711 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:57.203711 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:57.203711 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:57.203711 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:57.203711 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:57.203711 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:29:57.204606 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:29:57.204606 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:57.204606 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:57.211345 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 317/629] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:29:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:57.212561 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81f8e4-317b-444c-877f-4c000d7d1f8a None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:57.212900 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81f8e4-317b-444c-877f-4c000d7d1f8a None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:57.213110 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81f8e4-317b-444c-877f-4c000d7d1f8a None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:57.219640 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 318/630] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:57.219640 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c81f8e4-317b-444c-877f-4c000d7d1f8a None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:57.281145 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:57.284276 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e95851-d791-434d-86e2-e50da580562a None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:57.319871 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:57.326963 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdd0845-bd7f-4f4a-8e32-d09f135348be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:57.327709 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd0845-bd7f-4f4a-8e32-d09f135348be None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:57.327982 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd0845-bd7f-4f4a-8e32-d09f135348be None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:57.328252 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd0845-bd7f-4f4a-8e32-d09f135348be None admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:57.328892 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdd0845-bd7f-4f4a-8e32-d09f135348be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:57.329190 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdd0845-bd7f-4f4a-8e32-d09f135348be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:57.330763 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdd0845-bd7f-4f4a-8e32-d09f135348be None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:57.331319 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c81f8e4-317b-444c-877f-4c000d7d1f8a None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:57.348728 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 313/631] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:29:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:57.348728 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdd0845-bd7f-4f4a-8e32-d09f135348be None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:57.371160 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9316e427-dcab-422b-b32b-55904fd6158c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:57.371865 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9316e427-dcab-422b-b32b-55904fd6158c None admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:57.372150 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9316e427-dcab-422b-b32b-55904fd6158c None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:57.372421 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9316e427-dcab-422b-b32b-55904fd6158c None admin] PATH_INFO: `/v3/projects/01cf71ca5abf4576b89d10c85af6d2de/users/a18b13e3708144e1b5e8df80ec3a4e83/roles/1316db16eac54629a283545f1601a236` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:57.388675 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9316e427-dcab-422b-b32b-55904fd6158c None admin] RBAC: Authorizing `identity:create_grant(project_id=01cf71ca5abf4576b89d10c85af6d2de, user_id=a18b13e3708144e1b5e8df80ec3a4e83, role_id=1316db16eac54629a283545f1601a236)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:57.390559 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9316e427-dcab-422b-b32b-55904fd6158c None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:57.404366 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 319/632] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 30 14:29:57 2022] PUT /identity/v3/projects/01cf71ca5abf4576b89d10c85af6d2de/users/a18b13e3708144e1b5e8df80ec3a4e83/roles/1316db16eac54629a283545f1601a236 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:29:57.408668 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb6126-df85-402b-b751-d3da13560632 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:57.408959 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb6126-df85-402b-b751-d3da13560632 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:57.409263 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb6126-df85-402b-b751-d3da13560632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:57.432121 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-c0eb6126-df85-402b-b751-d3da13560632 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:57.433738 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-c0eb6126-df85-402b-b751-d3da13560632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:57.465876 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdd0845-bd7f-4f4a-8e32-d09f135348be None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:57.506760 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0631a9b9-eac7-48a0-bff4-ae5871545556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:57.507419 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0631a9b9-eac7-48a0-bff4-ae5871545556 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:57.507684 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0631a9b9-eac7-48a0-bff4-ae5871545556 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:57.507953 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0631a9b9-eac7-48a0-bff4-ae5871545556 None admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:57.508607 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0631a9b9-eac7-48a0-bff4-ae5871545556 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:57.508909 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0631a9b9-eac7-48a0-bff4-ae5871545556 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:57.510553 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0631a9b9-eac7-48a0-bff4-ae5871545556 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:57.511224 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 320/633] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:57.511901 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eb6126-df85-402b-b751-d3da13560632 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:57.523895 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 314/634] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:29:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:57.525141 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7195ab08-67a0-4de8-9ed7-73e086511a3c None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:57.525419 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7195ab08-67a0-4de8-9ed7-73e086511a3c None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:57.525685 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7195ab08-67a0-4de8-9ed7-73e086511a3c None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:57.535410 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 315/635] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:57.535410 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be44f9-8103-44e2-be1a-5ff742209e02 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:57.535410 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be44f9-8103-44e2-be1a-5ff742209e02 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:57.535410 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be44f9-8103-44e2-be1a-5ff742209e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:57.553141 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-03be44f9-8103-44e2-be1a-5ff742209e02 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:57.554357 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-03be44f9-8103-44e2-be1a-5ff742209e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:57.570758 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:57.619841 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:57.648654 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 316/636] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:29:57.649782 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be44f9-8103-44e2-be1a-5ff742209e02 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:57.666657 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eb6126-df85-402b-b751-d3da13560632 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:57.739415 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097e178c-3b80-4bf0-84ab-aa7261eab273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:57.740301 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e178c-3b80-4bf0-84ab-aa7261eab273 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:57.740752 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e178c-3b80-4bf0-84ab-aa7261eab273 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:57.741166 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e178c-3b80-4bf0-84ab-aa7261eab273 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:57.752982 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097e178c-3b80-4bf0-84ab-aa7261eab273 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:57.755706 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097e178c-3b80-4bf0-84ab-aa7261eab273 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:57.768928 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-097e178c-3b80-4bf0-84ab-aa7261eab273 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:29:57.768928 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:57.768928 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:57.768928 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:57.768928 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:57.768928 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:57.768928 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:57.768928 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:57.768928 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:57.768928 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:57.768928 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:57.768928 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:57.768928 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:29:57.768928 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:29:57.768928 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:29:57.768928 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:29:57.770118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:57.770118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:57.770118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:29:57.770118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:29:57.770118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:29:57.770118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:29:57.770118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:57.770118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:57.770118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:57.770118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:57.770118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:57.770118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:57.770118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:57.770118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:57.770118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:57.770118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:57.770118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:57.770118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:57.771297 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:57.771297 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:57.771297 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:29:57.771297 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:29:57.771297 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:29:57.771297 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:29:57.771297 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:29:57.771297 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:29:57.771297 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:29:57.771297 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:57.773462 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 321/637] 10.222.0.30 () {56 vars in 1093 bytes} [Tue Aug 30 14:29:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:57.779655 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097e178c-3b80-4bf0-84ab-aa7261eab273 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:57.829426 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be44f9-8103-44e2-be1a-5ff742209e02 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:57.864866 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5a5865-5fe3-4aff-b167-ca199e8369a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:57.865553 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5a5865-5fe3-4aff-b167-ca199e8369a5 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:57.865838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5a5865-5fe3-4aff-b167-ca199e8369a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:57.866112 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5a5865-5fe3-4aff-b167-ca199e8369a5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:57.874271 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5a5865-5fe3-4aff-b167-ca199e8369a5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:57.875748 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5a5865-5fe3-4aff-b167-ca199e8369a5 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:57.881764 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:57.888747 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-5d5a5865-5fe3-4aff-b167-ca199e8369a5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 30 14:29:57.888747 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:57.888747 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:57.888747 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:57.888747 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:57.888747 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:57.888747 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:57.888747 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:57.888747 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:57.888747 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:57.888747 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:57.888747 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:57.888747 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:29:57.888747 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:29:57.888747 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:29:57.888747 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:29:57.889918 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:57.889918 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:57.889918 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:29:57.889918 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:29:57.889918 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:29:57.889918 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:29:57.889918 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:57.889918 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:57.889918 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:57.889918 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:57.889918 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:57.889918 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:57.889918 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:57.889918 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:57.889918 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:57.889918 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:57.889918 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:57.889918 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:57.891338 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:57.891338 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:57.891338 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:29:57.891338 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:29:57.891338 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:29:57.891338 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:29:57.891338 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:29:57.891338 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:29:57.891338 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 30 14:29:57.891338 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:57.891338 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 317/638] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:29:57 2022] GET /identity/v3/users/glance => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:57.891338 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5a5865-5fe3-4aff-b167-ca199e8369a5 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:57.894343 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097e178c-3b80-4bf0-84ab-aa7261eab273 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:57.930021 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf29da-a8b9-4cef-b736-b7158c374651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:57.930859 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf29da-a8b9-4cef-b736-b7158c374651 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:57.931316 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf29da-a8b9-4cef-b736-b7158c374651 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:57.931731 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf29da-a8b9-4cef-b736-b7158c374651 None admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:57.933239 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bf29da-a8b9-4cef-b736-b7158c374651 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:57.934124 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bf29da-a8b9-4cef-b736-b7158c374651 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:57.935726 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bf29da-a8b9-4cef-b736-b7158c374651 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:57.936452 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:57.955408 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 322/639] 10.222.0.30 () {56 vars in 1133 bytes} [Tue Aug 30 14:29:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:57.956538 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf29da-a8b9-4cef-b736-b7158c374651 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:58.003328 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5a5865-5fe3-4aff-b167-ca199e8369a5 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:58.043489 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f45f2a-1061-4a68-86b7-17fb8345bc9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:58.044129 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f45f2a-1061-4a68-86b7-17fb8345bc9e None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:58.044766 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f45f2a-1061-4a68-86b7-17fb8345bc9e None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:58.045038 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f45f2a-1061-4a68-86b7-17fb8345bc9e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:58.055646 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f45f2a-1061-4a68-86b7-17fb8345bc9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:58.061283 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-d7f45f2a-1061-4a68-86b7-17fb8345bc9e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:29:58.061283 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:58.061283 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:58.061283 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:58.061283 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:58.061283 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:58.061283 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:58.061283 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:58.061283 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:58.061283 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:58.061283 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:58.061283 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:58.061283 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:58.061283 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:58.061283 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:58.061283 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:58.062224 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:58.062224 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:58.062224 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:58.062224 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:58.062224 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:58.062224 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:58.062224 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:58.062224 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:58.062224 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:58.062224 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:58.062224 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:58.062224 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:58.062224 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:58.062224 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:58.062224 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:58.062224 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:58.063053 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:58.063053 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:58.063053 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:58.063053 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:58.063053 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:58.063053 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:58.063053 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:58.063053 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:58.063053 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:58.063053 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:58.063053 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:29:58.063053 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:58.063053 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 318/640] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 30 14:29:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:58.063772 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f45f2a-1061-4a68-86b7-17fb8345bc9e None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:58.068119 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf29da-a8b9-4cef-b736-b7158c374651 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:58.102360 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d901e-9ae4-444a-abb1-2162ac55e6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:58.103227 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d901e-9ae4-444a-abb1-2162ac55e6e7 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:58.103511 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d901e-9ae4-444a-abb1-2162ac55e6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:58.103778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d901e-9ae4-444a-abb1-2162ac55e6e7 None admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:58.105751 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714d901e-9ae4-444a-abb1-2162ac55e6e7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:58.105751 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714d901e-9ae4-444a-abb1-2162ac55e6e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:58.106953 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714d901e-9ae4-444a-abb1-2162ac55e6e7 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:58.129466 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 323/641] 10.222.0.30 () {56 vars in 1137 bytes} [Tue Aug 30 14:29:57 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:58.129466 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d901e-9ae4-444a-abb1-2162ac55e6e7 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:58.139550 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:58.181898 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:58.186855 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f45f2a-1061-4a68-86b7-17fb8345bc9e None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:58.231874 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2798eb1d-acf5-400e-b3c1-dfc4809469d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:58.232592 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798eb1d-acf5-400e-b3c1-dfc4809469d6 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:58.232873 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798eb1d-acf5-400e-b3c1-dfc4809469d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:58.233146 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798eb1d-acf5-400e-b3c1-dfc4809469d6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:58.235123 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d901e-9ae4-444a-abb1-2162ac55e6e7 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:58.243789 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2798eb1d-acf5-400e-b3c1-dfc4809469d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:29:58.249223 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-2798eb1d-acf5-400e-b3c1-dfc4809469d6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:29:58.249223 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:58.249223 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:58.249223 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:58.249223 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:58.249223 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:58.249223 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:58.249223 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:58.249223 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:58.249223 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:58.249223 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:58.249223 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:58.249223 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:29:58.249223 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:29:58.249223 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:29:58.249223 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:29:58.250215 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:29:58.250215 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:29:58.250215 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:29:58.250215 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:29:58.250215 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:58.250215 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:58.250215 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:58.250215 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:58.250215 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:58.250215 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:58.250215 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:58.250215 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:58.250215 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:58.250215 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:58.250215 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:58.250215 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:58.251051 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:58.251051 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:58.251051 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:58.251051 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:58.251051 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:29:58.251051 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:29:58.251051 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:29:58.251051 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:29:58.251051 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:29:58.251051 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:29:58.251051 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:29:58.251051 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:58.251051 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 319/642] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 30 14:29:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:58.251777 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2798eb1d-acf5-400e-b3c1-dfc4809469d6 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:58.277576 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febc4c02-9c6b-43b8-9c9c-187e0500aa6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:58.278229 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc4c02-9c6b-43b8-9c9c-187e0500aa6f None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:58.278443 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc4c02-9c6b-43b8-9c9c-187e0500aa6f None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:58.278762 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc4c02-9c6b-43b8-9c9c-187e0500aa6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:58.349088 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febc4c02-9c6b-43b8-9c9c-187e0500aa6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:58.350527 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febc4c02-9c6b-43b8-9c9c-187e0500aa6f None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:58.359036 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2798eb1d-acf5-400e-b3c1-dfc4809469d6 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:58.363745 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:58.395016 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53727b23-452a-4fcb-bfce-29eb2060ec05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:58.395652 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53727b23-452a-4fcb-bfce-29eb2060ec05 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:58.395928 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53727b23-452a-4fcb-bfce-29eb2060ec05 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:58.396194 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53727b23-452a-4fcb-bfce-29eb2060ec05 None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:58.396845 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53727b23-452a-4fcb-bfce-29eb2060ec05 None admin] RBAC: Adding query filter params (name=service) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:58.397140 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53727b23-452a-4fcb-bfce-29eb2060ec05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:58.398489 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53727b23-452a-4fcb-bfce-29eb2060ec05 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:58.411689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 320/643] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 30 14:29:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:58.411689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53727b23-452a-4fcb-bfce-29eb2060ec05 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:58.434486 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:58.468636 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 324/644] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:29:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:29:58.469556 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febc4c02-9c6b-43b8-9c9c-187e0500aa6f None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:58.518337 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53727b23-452a-4fcb-bfce-29eb2060ec05 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:58.554748 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ce94bd-6863-4004-ae1f-b3463be78085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:58.555787 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce94bd-6863-4004-ae1f-b3463be78085 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:58.556271 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce94bd-6863-4004-ae1f-b3463be78085 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:58.556803 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce94bd-6863-4004-ae1f-b3463be78085 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:58.576861 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febc4c02-9c6b-43b8-9c9c-187e0500aa6f None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:58.589788 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:58.615931 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1a7824-e6e1-445b-95b7-31ddba3f9672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:58.616560 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a7824-e6e1-445b-95b7-31ddba3f9672 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:58.616895 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a7824-e6e1-445b-95b7-31ddba3f9672 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:58.617168 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a7824-e6e1-445b-95b7-31ddba3f9672 None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:58.617826 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1a7824-e6e1-445b-95b7-31ddba3f9672 None admin] RBAC: Adding query filter params (name=service) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:58.618181 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1a7824-e6e1-445b-95b7-31ddba3f9672 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:58.619607 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1a7824-e6e1-445b-95b7-31ddba3f9672 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:58.632766 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 325/645] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 30 14:29:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:58.632766 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1a7824-e6e1-445b-95b7-31ddba3f9672 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:58.642613 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ce94bd-6863-4004-ae1f-b3463be78085 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:58.644270 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ce94bd-6863-4004-ae1f-b3463be78085 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:58.668162 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:58.737695 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1a7824-e6e1-445b-95b7-31ddba3f9672 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:58.749896 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 321/646] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:29:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:29:58.750888 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ce94bd-6863-4004-ae1f-b3463be78085 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:58.778849 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb4c0d-a073-429a-b210-3a962e1bc8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:58.779768 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb4c0d-a073-429a-b210-3a962e1bc8d1 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:58.780043 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb4c0d-a073-429a-b210-3a962e1bc8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:58.780315 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb4c0d-a073-429a-b210-3a962e1bc8d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:58.791173 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eb4c0d-a073-429a-b210-3a962e1bc8d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:58.792949 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eb4c0d-a073-429a-b210-3a962e1bc8d1 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:58.808356 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-04eb4c0d-a073-429a-b210-3a962e1bc8d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:58.808356 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:58.808356 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:58.808356 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:58.808356 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:58.808356 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:58.808356 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:58.808356 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:58.808356 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:58.808356 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:58.808356 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:58.808356 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:58.808356 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:29:58.808356 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:29:58.808356 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:29:58.808356 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:29:58.809656 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:58.809656 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:58.809656 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:58.809656 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:58.809656 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:58.809656 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:58.809656 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:58.809656 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:58.809656 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:58.809656 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:58.809656 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:58.809656 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:58.809656 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:58.809656 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:58.809656 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:58.809656 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:58.809656 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:29:58.810613 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:29:58.810613 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:58.810613 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:58.810613 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 326/647] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:29:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:58.810847 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb4c0d-a073-429a-b210-3a962e1bc8d1 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:58.837929 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:58.862768 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ce94bd-6863-4004-ae1f-b3463be78085 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:58.887634 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:58.900676 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a87c600-29fc-4e66-b906-6a338d84c47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:58.901336 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87c600-29fc-4e66-b906-6a338d84c47a None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:58.901708 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87c600-29fc-4e66-b906-6a338d84c47a None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:58.902203 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87c600-29fc-4e66-b906-6a338d84c47a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:58.912970 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a87c600-29fc-4e66-b906-6a338d84c47a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:58.914406 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a87c600-29fc-4e66-b906-6a338d84c47a None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:58.921316 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb4c0d-a073-429a-b210-3a962e1bc8d1 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:58.930441 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-8a87c600-29fc-4e66-b906-6a338d84c47a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:58.930441 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:58.930441 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:58.930441 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:58.930441 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:58.930441 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:58.930441 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:58.930441 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:58.930441 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:58.930441 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:58.930441 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:58.930441 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:58.930441 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:29:58.930441 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:29:58.930441 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:29:58.930441 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:29:58.931430 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:58.931430 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:58.931430 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:58.931430 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:58.931430 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:58.931430 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:58.931430 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:58.931430 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:58.931430 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:58.931430 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:58.931430 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:58.931430 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:58.931430 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:58.931430 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:58.931430 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:58.931430 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:58.931430 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:29:58.932318 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:29:58.932318 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:58.932318 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:58.932318 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 322/648] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:29:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:58.932318 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a87c600-29fc-4e66-b906-6a338d84c47a None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:58.962549 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d5b674-7e8f-4b48-92bf-338246c7421f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:58.963185 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5b674-7e8f-4b48-92bf-338246c7421f None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:58.963501 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5b674-7e8f-4b48-92bf-338246c7421f None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:58.963812 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5b674-7e8f-4b48-92bf-338246c7421f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:58.975356 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d5b674-7e8f-4b48-92bf-338246c7421f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:58.976804 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d5b674-7e8f-4b48-92bf-338246c7421f None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:58.992498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-92d5b674-7e8f-4b48-92bf-338246c7421f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:58.992498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:58.992498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:58.992498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:58.992498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:58.992498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:58.992498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:58.992498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:58.992498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:58.992498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:58.992498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:58.992498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:58.992498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:29:58.992498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:29:58.992498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:29:58.992498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:29:58.993459 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:58.993459 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:58.993459 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:58.993459 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:58.993459 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:58.993459 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:58.993459 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:58.993459 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:58.993459 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:58.993459 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:58.993459 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:58.993459 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:58.993459 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:58.993459 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:58.993459 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:58.993459 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:58.993459 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:29:58.994376 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:29:58.994376 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:58.994376 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:58.994376 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 327/649] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:29:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:58.994376 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d5b674-7e8f-4b48-92bf-338246c7421f None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:59.044008 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a87c600-29fc-4e66-b906-6a338d84c47a None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:59.078914 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9454bc92-676c-47d4-a5df-f2d18116337a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:59.079577 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454bc92-676c-47d4-a5df-f2d18116337a None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:59.079853 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454bc92-676c-47d4-a5df-f2d18116337a None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:59.080120 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454bc92-676c-47d4-a5df-f2d18116337a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:59.081806 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:59.092417 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9454bc92-676c-47d4-a5df-f2d18116337a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:59.093955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9454bc92-676c-47d4-a5df-f2d18116337a None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:59.108137 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-9454bc92-676c-47d4-a5df-f2d18116337a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:59.108137 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:59.108137 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:59.108137 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:59.108137 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:59.108137 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:59.108137 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:59.108137 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:59.108137 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:59.108137 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:59.108137 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:59.108137 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:59.108137 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:29:59.108137 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:29:59.108137 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:29:59.108137 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:29:59.109627 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:59.109627 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:59.109627 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:59.109627 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:59.109627 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:59.109627 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:59.109627 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:59.109627 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:59.109627 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:59.109627 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:59.109627 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:59.109627 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:59.109627 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:59.109627 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:59.109627 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:59.109627 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:59.109627 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:29:59.115313 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:29:59.115313 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:29:59.115313 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:59.115538 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d5b674-7e8f-4b48-92bf-338246c7421f None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:59.115627 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 323/650] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:29:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:59.115627 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9454bc92-676c-47d4-a5df-f2d18116337a None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:59.145638 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:59.154933 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36212712-fed3-4580-9c83-01a6506456c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:59.155573 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36212712-fed3-4580-9c83-01a6506456c6 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:59.155857 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36212712-fed3-4580-9c83-01a6506456c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:59.156131 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36212712-fed3-4580-9c83-01a6506456c6 None admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:59.156765 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36212712-fed3-4580-9c83-01a6506456c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:59.157067 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36212712-fed3-4580-9c83-01a6506456c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:59.158426 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36212712-fed3-4580-9c83-01a6506456c6 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:59.175709 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 328/651] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:29:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:59.175709 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36212712-fed3-4580-9c83-01a6506456c6 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:59.239237 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9454bc92-676c-47d4-a5df-f2d18116337a None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:59.278607 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec90a622-f6e3-496b-8afa-652d692df8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:59.279065 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90a622-f6e3-496b-8afa-652d692df8a9 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:59.279405 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36212712-fed3-4580-9c83-01a6506456c6 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:59.279518 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90a622-f6e3-496b-8afa-652d692df8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:59.279909 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90a622-f6e3-496b-8afa-652d692df8a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:59.280876 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec90a622-f6e3-496b-8afa-652d692df8a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:59.281226 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec90a622-f6e3-496b-8afa-652d692df8a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:59.282654 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec90a622-f6e3-496b-8afa-652d692df8a9 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:59.295819 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 324/652] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:29:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:59.296638 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec90a622-f6e3-496b-8afa-652d692df8a9 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:59.321612 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d71e07-2fc3-41f6-a161-5244e7d483e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:59.322223 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d71e07-2fc3-41f6-a161-5244e7d483e8 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:59.322502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d71e07-2fc3-41f6-a161-5244e7d483e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:59.322793 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d71e07-2fc3-41f6-a161-5244e7d483e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:59.323998 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d71e07-2fc3-41f6-a161-5244e7d483e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:59.324296 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d71e07-2fc3-41f6-a161-5244e7d483e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:59.325671 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d71e07-2fc3-41f6-a161-5244e7d483e8 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:59.328028 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:59.339466 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 329/653] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:29:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:59.344829 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d71e07-2fc3-41f6-a161-5244e7d483e8 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:59.392419 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:59.409499 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec90a622-f6e3-496b-8afa-652d692df8a9 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:59.447123 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4849d133-1b89-48f7-833d-9dc6b093bfc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:59.447745 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849d133-1b89-48f7-833d-9dc6b093bfc1 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:59.448023 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849d133-1b89-48f7-833d-9dc6b093bfc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:59.448942 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849d133-1b89-48f7-833d-9dc6b093bfc1 None admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:59.449592 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4849d133-1b89-48f7-833d-9dc6b093bfc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:59.449897 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4849d133-1b89-48f7-833d-9dc6b093bfc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:59.451272 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4849d133-1b89-48f7-833d-9dc6b093bfc1 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:59.453671 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d71e07-2fc3-41f6-a161-5244e7d483e8 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:59.462136 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 325/654] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:29:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:59.462924 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4849d133-1b89-48f7-833d-9dc6b093bfc1 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:59.491239 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376f558e-627c-4f64-8dd3-b3e0a3a14ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:59.491869 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f558e-627c-4f64-8dd3-b3e0a3a14ad3 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:59.492145 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f558e-627c-4f64-8dd3-b3e0a3a14ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:59.492415 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f558e-627c-4f64-8dd3-b3e0a3a14ad3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:59.500735 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376f558e-627c-4f64-8dd3-b3e0a3a14ad3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:59.502175 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376f558e-627c-4f64-8dd3-b3e0a3a14ad3 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:59.513625 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-376f558e-627c-4f64-8dd3-b3e0a3a14ad3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 14:29:59.513625 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:59.513625 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:59.513625 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:59.513625 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:59.513625 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:59.513625 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:59.513625 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:59.513625 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:59.513625 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:59.513625 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:59.513625 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:59.513625 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:29:59.513625 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:29:59.513625 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:29:59.513625 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:29:59.514602 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:59.514602 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:59.514602 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:29:59.514602 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:29:59.514602 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:29:59.514602 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:29:59.514602 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:59.514602 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:59.514602 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:59.514602 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:59.514602 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:59.514602 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:59.514602 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:59.514602 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:59.514602 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:59.514602 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:59.514602 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:59.514602 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:59.515561 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:59.515561 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:59.515561 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:29:59.515561 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:29:59.515561 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:29:59.515561 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:29:59.515561 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:29:59.515561 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:29:59.515561 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 14:29:59.515561 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:59.515561 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 330/655] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 30 14:29:59 2022] GET /identity/v3/users/neutron => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:59.516149 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376f558e-627c-4f64-8dd3-b3e0a3a14ad3 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:59.566139 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4849d133-1b89-48f7-833d-9dc6b093bfc1 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:59.570129 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:59.600522 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ac26dd-8e71-42a0-adef-ca64649f62f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:59.601143 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ac26dd-8e71-42a0-adef-ca64649f62f8 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:59.601424 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ac26dd-8e71-42a0-adef-ca64649f62f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:59.601694 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ac26dd-8e71-42a0-adef-ca64649f62f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:59.608275 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ac26dd-8e71-42a0-adef-ca64649f62f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:59.609720 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ac26dd-8e71-42a0-adef-ca64649f62f8 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:59.616046 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376f558e-627c-4f64-8dd3-b3e0a3a14ad3 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:59.621137 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-e4ac26dd-8e71-42a0-adef-ca64649f62f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:29:59.621137 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:59.621137 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:59.621137 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:59.621137 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:59.621137 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:59.621137 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:59.621137 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:59.621137 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:59.621137 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:59.621137 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:59.621137 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:59.621137 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:29:59.621137 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:29:59.621137 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:29:59.622052 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:29:59.622052 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:59.622052 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:59.622052 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:59.622052 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:59.622052 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:59.622052 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:59.622052 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:59.622052 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:59.622052 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:59.622052 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:59.622052 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:59.622052 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:59.622052 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:59.622052 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:59.622052 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:59.622052 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:59.622943 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:29:59.622943 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:29:59.622943 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:29:59.622943 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:29:59.622943 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:29:59.622943 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:29:59.622943 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:29:59.622943 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:59.622943 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 326/656] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:29:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:59.623475 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebafd5d0-bd63-4289-a42c-76edae7cb00c None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:59.623679 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebafd5d0-bd63-4289-a42c-76edae7cb00c None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:59.623956 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebafd5d0-bd63-4289-a42c-76edae7cb00c None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:59.625107 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 327/657] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:29:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:29:59.625791 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebafd5d0-bd63-4289-a42c-76edae7cb00c None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:59.637232 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:59.656141 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db86327-34b0-4f94-959e-1a2761584ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:59.656765 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db86327-34b0-4f94-959e-1a2761584ccc None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:59.657045 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db86327-34b0-4f94-959e-1a2761584ccc None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:59.657313 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db86327-34b0-4f94-959e-1a2761584ccc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:59.664579 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db86327-34b0-4f94-959e-1a2761584ccc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:59.666002 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db86327-34b0-4f94-959e-1a2761584ccc None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:59.676985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-5db86327-34b0-4f94-959e-1a2761584ccc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 14:29:59.676985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:59.676985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:59.676985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:59.676985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:59.676985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:59.676985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:59.676985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:59.676985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:59.676985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:59.676985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:59.676985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:59.676985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:29:59.676985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:29:59.676985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:29:59.676985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:29:59.677899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:59.677899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:59.677899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:29:59.677899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:29:59.677899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:29:59.677899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:29:59.677899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:59.677899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:59.677899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:59.677899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:59.677899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:59.677899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:59.677899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:59.677899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:59.677899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:59.677899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:59.677899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:59.677899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:59.678834 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:59.678834 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:59.678834 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:29:59.678834 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:29:59.678834 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:29:59.678834 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:29:59.678834 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:29:59.678834 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:29:59.678834 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 14:29:59.678834 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:29:59.678834 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 331/658] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:29:59 2022] GET /identity/v3/users/cinder => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:59.678834 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db86327-34b0-4f94-959e-1a2761584ccc None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:59.736313 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebafd5d0-bd63-4289-a42c-76edae7cb00c None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:59.769825 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d973d33d-ce12-4b71-886b-de2d2e60eb80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:59.770450 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973d33d-ce12-4b71-886b-de2d2e60eb80 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:59.770733 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973d33d-ce12-4b71-886b-de2d2e60eb80 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:59.771005 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973d33d-ce12-4b71-886b-de2d2e60eb80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:59.777708 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d973d33d-ce12-4b71-886b-de2d2e60eb80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:59.779172 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d973d33d-ce12-4b71-886b-de2d2e60eb80 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:59.781375 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db86327-34b0-4f94-959e-1a2761584ccc None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:59.790575 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-d973d33d-ce12-4b71-886b-de2d2e60eb80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:29:59.790575 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:29:59.790575 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:29:59.790575 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:29:59.790575 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:29:59.790575 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:29:59.790575 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:29:59.790575 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:29:59.790575 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:29:59.790575 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:29:59.790575 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:29:59.790575 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:29:59.790575 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:29:59.790575 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:29:59.790575 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:29:59.791629 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:29:59.791629 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:29:59.791629 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:29:59.791629 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:29:59.791629 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:29:59.791629 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:29:59.791629 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:29:59.791629 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:29:59.791629 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:29:59.791629 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:29:59.791629 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:29:59.791629 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:29:59.791629 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:29:59.791629 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:29:59.791629 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:29:59.791629 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:29:59.791629 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:29:59.793988 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:29:59.793988 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:29:59.793988 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:29:59.793988 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:29:59.793988 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:29:59.793988 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:29:59.793988 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:29:59.793988 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:29:59.793988 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 328/659] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:29:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:29:59.793988 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d973d33d-ce12-4b71-886b-de2d2e60eb80 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:59.800514 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:59.820738 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dbb384-6391-4c20-ab21-756d64c9222b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:59.821269 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dbb384-6391-4c20-ab21-756d64c9222b None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:59.821548 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dbb384-6391-4c20-ab21-756d64c9222b None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:59.821815 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dbb384-6391-4c20-ab21-756d64c9222b None admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:59.822536 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dbb384-6391-4c20-ab21-756d64c9222b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:59.822837 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dbb384-6391-4c20-ab21-756d64c9222b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:59.824432 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dbb384-6391-4c20-ab21-756d64c9222b None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:59.841095 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 332/660] 10.222.0.30 () {56 vars in 1139 bytes} [Tue Aug 30 14:29:59 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:59.842294 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c63cd-d509-4b54-80d5-faa1fcc09513 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:59.842532 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c63cd-d509-4b54-80d5-faa1fcc09513 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:59.842795 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c63cd-d509-4b54-80d5-faa1fcc09513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:59.859988 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:29:59.860258 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-4a8c63cd-d509-4b54-80d5-faa1fcc09513 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:29:59.861440 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-4a8c63cd-d509-4b54-80d5-faa1fcc09513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:29:59.895646 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d973d33d-ce12-4b71-886b-de2d2e60eb80 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:29:59.926237 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 333/661] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:29:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:29:59.927062 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8c63cd-d509-4b54-80d5-faa1fcc09513 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:29:59.929146 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b7c02d-6b3f-46f6-a498-e92a7578c9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:29:59.929829 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7c02d-6b3f-46f6-a498-e92a7578c9cd None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:29:59.930181 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7c02d-6b3f-46f6-a498-e92a7578c9cd None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:29:59.930533 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7c02d-6b3f-46f6-a498-e92a7578c9cd None admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:29:59.931329 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b7c02d-6b3f-46f6-a498-e92a7578c9cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:29:59.931705 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b7c02d-6b3f-46f6-a498-e92a7578c9cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:29:59.934134 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b7c02d-6b3f-46f6-a498-e92a7578c9cd None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:29:59.947994 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 329/662] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 30 14:29:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:29:59.948787 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b7c02d-6b3f-46f6-a498-e92a7578c9cd None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:00.038000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8c63cd-d509-4b54-80d5-faa1fcc09513 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:00.041161 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:00.052501 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b7c02d-6b3f-46f6-a498-e92a7578c9cd None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:00.081086 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ba9eaa-d5d2-484f-b173-28ed2771e573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:00.081293 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:00.081608 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ba9eaa-d5d2-484f-b173-28ed2771e573 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:00.081884 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ba9eaa-d5d2-484f-b173-28ed2771e573 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:00.082249 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ba9eaa-d5d2-484f-b173-28ed2771e573 None admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:00.083002 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ba9eaa-d5d2-484f-b173-28ed2771e573 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:00.083186 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ba9eaa-d5d2-484f-b173-28ed2771e573 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:00.084612 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ba9eaa-d5d2-484f-b173-28ed2771e573 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:00.092243 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff20b84-942f-42a7-9812-23fb2d852976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:00.092855 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff20b84-942f-42a7-9812-23fb2d852976 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:00.093141 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff20b84-942f-42a7-9812-23fb2d852976 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:00.093421 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff20b84-942f-42a7-9812-23fb2d852976 None admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:00.094124 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff20b84-942f-42a7-9812-23fb2d852976 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:00.094429 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff20b84-942f-42a7-9812-23fb2d852976 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:00.095895 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff20b84-942f-42a7-9812-23fb2d852976 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:00.103864 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 334/663] 10.222.0.30 () {56 vars in 1137 bytes} [Tue Aug 30 14:29:59 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:00.104765 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ba9eaa-d5d2-484f-b173-28ed2771e573 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:00.117718 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 330/664] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 30 14:29:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:00.118450 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff20b84-942f-42a7-9812-23fb2d852976 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:00.216522 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ba9eaa-d5d2-484f-b173-28ed2771e573 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:00.235442 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff20b84-942f-42a7-9812-23fb2d852976 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:00.251585 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f84a6a-df7e-4cb4-b9ca-e18c58749aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:00.252197 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f84a6a-df7e-4cb4-b9ca-e18c58749aa4 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:00.252470 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f84a6a-df7e-4cb4-b9ca-e18c58749aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:00.252734 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f84a6a-df7e-4cb4-b9ca-e18c58749aa4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:00.271405 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bdb2b9-0e40-43a9-a517-ef8b227c3e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:00.272014 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bdb2b9-0e40-43a9-a517-ef8b227c3e81 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:00.272294 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bdb2b9-0e40-43a9-a517-ef8b227c3e81 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:00.272568 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bdb2b9-0e40-43a9-a517-ef8b227c3e81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:00.273288 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bdb2b9-0e40-43a9-a517-ef8b227c3e81 None admin] RBAC: Adding query filter params (role.id=1316db16eac54629a283545f1601a236, scope.project.id=01cf71ca5abf4576b89d10c85af6d2de, user.id=8878de66a03044e38e507028853a0cd1) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:00.273596 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bdb2b9-0e40-43a9-a517-ef8b227c3e81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:00.275015 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bdb2b9-0e40-43a9-a517-ef8b227c3e81 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:00.281196 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 331/665] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:30:00 2022] GET /identity/v3/role_assignments?user.id=8878de66a03044e38e507028853a0cd1&scope.project.id=01cf71ca5abf4576b89d10c85af6d2de&role.id=1316db16eac54629a283545f1601a236 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:00.282098 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bdb2b9-0e40-43a9-a517-ef8b227c3e81 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:00.284220 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:00.335347 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f84a6a-df7e-4cb4-b9ca-e18c58749aa4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:00.336884 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f84a6a-df7e-4cb4-b9ca-e18c58749aa4 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:00.340085 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:00.398458 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bdb2b9-0e40-43a9-a517-ef8b227c3e81 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:00.439699 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db1b320-51bb-4940-9c5e-dde26de37fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:00.440309 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db1b320-51bb-4940-9c5e-dde26de37fd2 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:00.440588 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db1b320-51bb-4940-9c5e-dde26de37fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:00.440855 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db1b320-51bb-4940-9c5e-dde26de37fd2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:00.452669 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 335/666] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:30:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:30:00.453871 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f84a6a-df7e-4cb4-b9ca-e18c58749aa4 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:00.522715 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db1b320-51bb-4940-9c5e-dde26de37fd2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:00.524221 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db1b320-51bb-4940-9c5e-dde26de37fd2 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:00.527901 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:00.568787 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f84a6a-df7e-4cb4-b9ca-e18c58749aa4 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:00.577613 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:00.608157 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5cfd18-6ea5-49b0-a95a-eaf1efe0765f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:00.608686 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5cfd18-6ea5-49b0-a95a-eaf1efe0765f None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:00.608965 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5cfd18-6ea5-49b0-a95a-eaf1efe0765f None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:00.609236 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5cfd18-6ea5-49b0-a95a-eaf1efe0765f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:00.609953 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5cfd18-6ea5-49b0-a95a-eaf1efe0765f None admin] RBAC: Adding query filter params (role.id=1316db16eac54629a283545f1601a236, scope.project.id=01cf71ca5abf4576b89d10c85af6d2de, user.id=6889402e54424bbd8d42c7bb759cf700) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:00.610379 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5cfd18-6ea5-49b0-a95a-eaf1efe0765f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:00.611943 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5cfd18-6ea5-49b0-a95a-eaf1efe0765f None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:00.627246 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 336/667] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:30:00 2022] GET /identity/v3/role_assignments?user.id=6889402e54424bbd8d42c7bb759cf700&scope.project.id=01cf71ca5abf4576b89d10c85af6d2de&role.id=1316db16eac54629a283545f1601a236 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:00.627246 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5cfd18-6ea5-49b0-a95a-eaf1efe0765f None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:00.642844 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 332/668] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:30:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:30:00.644016 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db1b320-51bb-4940-9c5e-dde26de37fd2 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:00.769584 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db1b320-51bb-4940-9c5e-dde26de37fd2 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:00.774675 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5cfd18-6ea5-49b0-a95a-eaf1efe0765f None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:00.811666 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:00.816038 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbff3ab8-ae42-4921-a4f6-5b5ceb724a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:00.816839 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff3ab8-ae42-4921-a4f6-5b5ceb724a6a None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:00.817214 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff3ab8-ae42-4921-a4f6-5b5ceb724a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:00.817682 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff3ab8-ae42-4921-a4f6-5b5ceb724a6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:00.821548 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a591e3d1-303f-4789-b45b-86dd32cb443d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:00.822184 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a591e3d1-303f-4789-b45b-86dd32cb443d None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:00.822481 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a591e3d1-303f-4789-b45b-86dd32cb443d None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:00.822756 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a591e3d1-303f-4789-b45b-86dd32cb443d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:00.829752 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbff3ab8-ae42-4921-a4f6-5b5ceb724a6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:00.831473 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbff3ab8-ae42-4921-a4f6-5b5ceb724a6a None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:00.845672 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a591e3d1-303f-4789-b45b-86dd32cb443d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:00.850999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-a591e3d1-303f-4789-b45b-86dd32cb443d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:30:00.850999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:00.850999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:00.850999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:00.850999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:00.850999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:00.850999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:00.850999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:00.850999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:00.850999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:00.850999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:00.850999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:00.850999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:00.850999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:00.850999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:00.850999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:00.852226 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:00.852226 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:00.852226 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:00.852226 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:00.852226 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:00.852226 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:00.852226 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:00.852226 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:00.852226 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:00.852226 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:00.852226 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:00.852226 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:00.852226 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:00.852226 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:00.852226 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:00.852226 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:00.853349 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:00.853349 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:00.853349 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:00.853349 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:00.853349 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:00.853349 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:00.853349 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:00.853349 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:00.853349 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:00.853349 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:00.853349 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:30:00.853349 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:00.856482 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 337/669] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 30 14:30:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:00.857525 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a591e3d1-303f-4789-b45b-86dd32cb443d None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:00.866401 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-bbff3ab8-ae42-4921-a4f6-5b5ceb724a6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:00.866401 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:00.866401 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:00.866401 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:00.866401 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:00.866401 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:00.866401 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:00.866401 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:00.866401 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:00.866401 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:00.866401 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:00.866401 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:00.866401 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:30:00.866401 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:30:00.866401 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:30:00.866401 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:30:00.867445 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:00.867445 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:00.867445 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:00.867445 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:00.867445 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:00.867445 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:00.867445 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:00.867445 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:00.867445 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:00.867445 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:00.867445 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:00.867445 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:00.867445 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:00.867445 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:00.867445 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:00.867445 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:00.867445 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:30:00.868343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:30:00.868343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:00.868343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:00.868343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 333/670] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:00.868580 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbff3ab8-ae42-4921-a4f6-5b5ceb724a6a None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:00.872934 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:00.994240 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbff3ab8-ae42-4921-a4f6-5b5ceb724a6a None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:01.003436 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a591e3d1-303f-4789-b45b-86dd32cb443d None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:01.035792 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0888e57e-ac89-4e80-aed4-28b257600d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:01.036665 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888e57e-ac89-4e80-aed4-28b257600d3d None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:01.036961 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888e57e-ac89-4e80-aed4-28b257600d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:01.037237 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888e57e-ac89-4e80-aed4-28b257600d3d None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:01.038414 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0888e57e-ac89-4e80-aed4-28b257600d3d None admin] RBAC: Adding query filter params (name=service) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:01.038731 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0888e57e-ac89-4e80-aed4-28b257600d3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:01.040395 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0888e57e-ac89-4e80-aed4-28b257600d3d None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:01.048786 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbccfe3-8bb3-4e5d-a3b7-9543ab15409e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:01.049708 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbccfe3-8bb3-4e5d-a3b7-9543ab15409e None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:01.050203 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbccfe3-8bb3-4e5d-a3b7-9543ab15409e None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:01.050718 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbccfe3-8bb3-4e5d-a3b7-9543ab15409e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:01.055178 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 334/671] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 30 14:30:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:01.056513 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0888e57e-ac89-4e80-aed4-28b257600d3d None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:01.067841 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbccfe3-8bb3-4e5d-a3b7-9543ab15409e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:01.070088 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbccfe3-8bb3-4e5d-a3b7-9543ab15409e None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:01.090585 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-2fbccfe3-8bb3-4e5d-a3b7-9543ab15409e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:01.090585 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:01.090585 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:01.090585 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:01.090585 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:01.090585 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:01.090585 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:01.090585 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:01.090585 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:01.090585 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:01.090585 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:01.090585 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:01.090585 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:30:01.090585 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:30:01.090585 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:30:01.090585 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:30:01.091610 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:01.091610 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:01.091610 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:01.091610 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:01.091610 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:01.091610 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:01.091610 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:01.091610 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:01.091610 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:01.091610 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:01.091610 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:01.091610 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:01.091610 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:01.091610 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:01.091610 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:01.091610 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:01.091610 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:30:01.092571 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:30:01.092571 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:01.092571 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:01.093267 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 338/672] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:01.093779 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbccfe3-8bb3-4e5d-a3b7-9543ab15409e None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:01.143168 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:01.147402 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:01.188019 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0888e57e-ac89-4e80-aed4-28b257600d3d None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:01.237247 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee380d0-3ea5-4d45-89ef-a54d5ba88e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:01.238595 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee380d0-3ea5-4d45-89ef-a54d5ba88e7f None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:01.239298 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee380d0-3ea5-4d45-89ef-a54d5ba88e7f None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:01.239734 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee380d0-3ea5-4d45-89ef-a54d5ba88e7f None admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:01.240694 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee380d0-3ea5-4d45-89ef-a54d5ba88e7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:01.241512 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee380d0-3ea5-4d45-89ef-a54d5ba88e7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:01.243409 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee380d0-3ea5-4d45-89ef-a54d5ba88e7f None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:01.244066 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbccfe3-8bb3-4e5d-a3b7-9543ab15409e None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:01.260937 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 335/673] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:30:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:01.262297 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee380d0-3ea5-4d45-89ef-a54d5ba88e7f None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:01.295894 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261866a1-b50c-45a9-bfca-54a69c34ea6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:01.297073 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261866a1-b50c-45a9-bfca-54a69c34ea6c None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:01.297469 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261866a1-b50c-45a9-bfca-54a69c34ea6c None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:01.297839 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261866a1-b50c-45a9-bfca-54a69c34ea6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:01.313529 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261866a1-b50c-45a9-bfca-54a69c34ea6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:01.315354 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261866a1-b50c-45a9-bfca-54a69c34ea6c None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:01.333841 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-261866a1-b50c-45a9-bfca-54a69c34ea6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:01.333841 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:01.333841 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:01.333841 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:01.333841 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:01.333841 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:01.333841 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:01.333841 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:01.333841 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:01.333841 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:01.333841 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:01.333841 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:01.333841 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:30:01.333841 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:30:01.333841 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:30:01.333841 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:30:01.334994 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:01.334994 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:01.334994 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:01.334994 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:01.334994 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:01.334994 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:01.334994 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:01.334994 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:01.334994 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:01.334994 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:01.334994 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:01.334994 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:01.334994 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:01.334994 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:01.334994 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:01.334994 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:01.334994 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:30:01.335932 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:30:01.335932 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:01.335932 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:01.337879 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 339/674] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:01.338883 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261866a1-b50c-45a9-bfca-54a69c34ea6c None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:01.393126 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee380d0-3ea5-4d45-89ef-a54d5ba88e7f None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:01.420336 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:01.435063 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e82cb02-b938-4d70-90d0-f1928d16ee56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:01.436104 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82cb02-b938-4d70-90d0-f1928d16ee56 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:01.436608 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82cb02-b938-4d70-90d0-f1928d16ee56 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:01.437114 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82cb02-b938-4d70-90d0-f1928d16ee56 None admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:01.438025 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e82cb02-b938-4d70-90d0-f1928d16ee56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:01.438544 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e82cb02-b938-4d70-90d0-f1928d16ee56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:01.440476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e82cb02-b938-4d70-90d0-f1928d16ee56 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:01.453915 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 336/675] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:30:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:01.455148 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e82cb02-b938-4d70-90d0-f1928d16ee56 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:01.461550 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:01.473629 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261866a1-b50c-45a9-bfca-54a69c34ea6c None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:01.511859 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e67ad02-3e53-451a-9a34-c8e94ec62420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:01.512802 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67ad02-3e53-451a-9a34-c8e94ec62420 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:01.513320 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67ad02-3e53-451a-9a34-c8e94ec62420 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:01.513813 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67ad02-3e53-451a-9a34-c8e94ec62420 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:01.521605 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e67ad02-3e53-451a-9a34-c8e94ec62420 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:01.523325 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e67ad02-3e53-451a-9a34-c8e94ec62420 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:01.535144 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-9e67ad02-3e53-451a-9a34-c8e94ec62420 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:30:01.535144 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:01.535144 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:01.535144 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:01.535144 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:01.535144 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:01.535144 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:01.535144 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:01.535144 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:01.535144 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:01.535144 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:01.535144 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:01.535144 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:30:01.535144 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:30:01.535144 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:30:01.536064 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:30:01.536064 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:01.536064 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:01.536064 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:01.536064 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:01.536064 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:01.536064 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:01.536064 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:01.536064 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:01.536064 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:01.536064 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:01.536064 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:01.536064 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:01.536064 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:01.536064 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:01.536064 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:01.536064 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:01.536952 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:30:01.536952 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:30:01.536952 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:30:01.536952 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:30:01.536952 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:30:01.536952 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:30:01.536952 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:30:01.536952 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:01.539107 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 340/676] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:30:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:01.540242 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e67ad02-3e53-451a-9a34-c8e94ec62420 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:01.582308 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e82cb02-b938-4d70-90d0-f1928d16ee56 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:01.622108 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f408df73-efa1-4cb4-9022-1ba444bdf128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:01.622723 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408df73-efa1-4cb4-9022-1ba444bdf128 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:01.623005 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408df73-efa1-4cb4-9022-1ba444bdf128 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:01.623298 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408df73-efa1-4cb4-9022-1ba444bdf128 None admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:01.623944 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f408df73-efa1-4cb4-9022-1ba444bdf128 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:01.624244 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f408df73-efa1-4cb4-9022-1ba444bdf128 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:01.625606 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f408df73-efa1-4cb4-9022-1ba444bdf128 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:01.640337 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 337/677] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:30:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:01.641428 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f408df73-efa1-4cb4-9022-1ba444bdf128 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:01.674849 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e67ad02-3e53-451a-9a34-c8e94ec62420 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:01.695166 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:01.720129 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ed66e2-60ef-4562-a0ee-fe6562e5f16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:01.720769 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed66e2-60ef-4562-a0ee-fe6562e5f16e None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:01.721541 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed66e2-60ef-4562-a0ee-fe6562e5f16e None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:01.721818 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed66e2-60ef-4562-a0ee-fe6562e5f16e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:01.730844 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ed66e2-60ef-4562-a0ee-fe6562e5f16e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:01.732758 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ed66e2-60ef-4562-a0ee-fe6562e5f16e None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:01.747857 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-b9ed66e2-60ef-4562-a0ee-fe6562e5f16e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:30:01.747857 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:01.747857 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:01.747857 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:01.747857 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:01.747857 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:01.747857 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:01.747857 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:01.747857 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:01.747857 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:01.747857 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:01.747857 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:01.747857 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:30:01.747857 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:30:01.747857 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:30:01.748810 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:30:01.748810 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:01.748810 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:01.748810 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:01.748810 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:01.748810 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:01.748810 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:01.748810 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:01.748810 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:01.748810 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:01.748810 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:01.748810 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:01.748810 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:01.748810 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:01.748810 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:01.748810 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:01.748810 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:01.750333 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:30:01.750333 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:30:01.750333 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:30:01.750333 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:30:01.750333 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:30:01.750333 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:30:01.750333 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:30:01.750333 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:01.750333 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 341/678] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:30:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:01.750333 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ed66e2-60ef-4562-a0ee-fe6562e5f16e None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:01.763954 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f408df73-efa1-4cb4-9022-1ba444bdf128 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:01.768913 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:01.813918 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5837dc7-a0fa-43df-a45d-fbb9df0250ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:01.814989 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5837dc7-a0fa-43df-a45d-fbb9df0250ad None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:01.815941 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5837dc7-a0fa-43df-a45d-fbb9df0250ad None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:01.816408 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5837dc7-a0fa-43df-a45d-fbb9df0250ad None admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:01.817278 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5837dc7-a0fa-43df-a45d-fbb9df0250ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:01.818189 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5837dc7-a0fa-43df-a45d-fbb9df0250ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:01.820232 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5837dc7-a0fa-43df-a45d-fbb9df0250ad None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:01.843738 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 338/679] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 30 14:30:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:01.845134 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5837dc7-a0fa-43df-a45d-fbb9df0250ad None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:01.891286 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ed66e2-60ef-4562-a0ee-fe6562e5f16e None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:01.927931 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635be211-7f22-4e65-bda3-e3c569ab8738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:01.928495 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635be211-7f22-4e65-bda3-e3c569ab8738 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:01.928734 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635be211-7f22-4e65-bda3-e3c569ab8738 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:01.929004 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635be211-7f22-4e65-bda3-e3c569ab8738 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:01.937813 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635be211-7f22-4e65-bda3-e3c569ab8738 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:01.939485 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635be211-7f22-4e65-bda3-e3c569ab8738 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:01.952441 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-635be211-7f22-4e65-bda3-e3c569ab8738 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 14:30:01.952441 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:01.952441 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:01.952441 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:01.952441 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:01.952441 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:01.952441 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:01.952441 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:01.952441 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:01.952441 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:01.952441 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:01.952441 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:01.952441 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:30:01.952441 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:30:01.952441 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:30:01.952441 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:30:01.953397 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:01.953397 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:01.953397 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:30:01.953397 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:30:01.953397 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:30:01.953397 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:30:01.953397 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:01.953397 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:01.953397 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:01.953397 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:01.953397 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:01.953397 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:01.953397 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:01.953397 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:01.953397 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:01.953397 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:01.953397 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:01.953397 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:01.954415 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:01.954415 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:01.954415 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:30:01.954415 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:30:01.954415 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:30:01.954415 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:30:01.954415 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:30:01.954415 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:30:01.954415 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 14:30:01.954415 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:01.958029 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 342/680] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 30 14:30:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:01.958878 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635be211-7f22-4e65-bda3-e3c569ab8738 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:01.984248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5837dc7-a0fa-43df-a45d-fbb9df0250ad None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:01.987516 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:02.030492 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d489945e-b747-47be-939a-8e28f578112c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:02.031757 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489945e-b747-47be-939a-8e28f578112c None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:02.032290 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489945e-b747-47be-939a-8e28f578112c None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:02.032992 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489945e-b747-47be-939a-8e28f578112c None admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:02.034520 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d489945e-b747-47be-939a-8e28f578112c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:02.035028 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d489945e-b747-47be-939a-8e28f578112c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:02.036919 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d489945e-b747-47be-939a-8e28f578112c None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:02.041292 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:02.055101 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 339/681] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 30 14:30:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:02.055962 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d489945e-b747-47be-939a-8e28f578112c None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:02.085438 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635be211-7f22-4e65-bda3-e3c569ab8738 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:02.127925 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75df374-1acf-4601-a1cb-3c9fc17ad033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:02.128957 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75df374-1acf-4601-a1cb-3c9fc17ad033 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:02.129391 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75df374-1acf-4601-a1cb-3c9fc17ad033 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:02.130143 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75df374-1acf-4601-a1cb-3c9fc17ad033 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:02.131298 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75df374-1acf-4601-a1cb-3c9fc17ad033 None admin] RBAC: Adding query filter params (role.id=1316db16eac54629a283545f1601a236, scope.project.id=01cf71ca5abf4576b89d10c85af6d2de, user.id=9e14e86e085d4987aece69fe64867ce6) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:02.131763 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75df374-1acf-4601-a1cb-3c9fc17ad033 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:02.133785 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75df374-1acf-4601-a1cb-3c9fc17ad033 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:02.148196 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 343/682] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:30:01 2022] GET /identity/v3/role_assignments?user.id=9e14e86e085d4987aece69fe64867ce6&scope.project.id=01cf71ca5abf4576b89d10c85af6d2de&role.id=1316db16eac54629a283545f1601a236 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:02.148196 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75df374-1acf-4601-a1cb-3c9fc17ad033 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:02.218045 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d489945e-b747-47be-939a-8e28f578112c None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:02.312987 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d08ae-1d32-4eba-8191-794c45eb300f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:02.313628 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d08ae-1d32-4eba-8191-794c45eb300f None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:02.313913 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d08ae-1d32-4eba-8191-794c45eb300f None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:02.314188 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d08ae-1d32-4eba-8191-794c45eb300f None admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:02.314939 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4d08ae-1d32-4eba-8191-794c45eb300f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:02.315955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4d08ae-1d32-4eba-8191-794c45eb300f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:02.317916 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4d08ae-1d32-4eba-8191-794c45eb300f None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:02.340263 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 340/683] 10.222.0.30 () {56 vars in 1135 bytes} [Tue Aug 30 14:30:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:02.341880 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad05a3-e00d-43fd-998a-8f139c41ae57 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:02.342226 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad05a3-e00d-43fd-998a-8f139c41ae57 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:02.342578 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad05a3-e00d-43fd-998a-8f139c41ae57 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:02.344540 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 341/684] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:02.349431 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c92f7-27d4-4c26-ba34-00474190d3d2 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:02.349867 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c92f7-27d4-4c26-ba34-00474190d3d2 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:02.350109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c92f7-27d4-4c26-ba34-00474190d3d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:02.357298 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75df374-1acf-4601-a1cb-3c9fc17ad033 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:02.373400 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-696c92f7-27d4-4c26-ba34-00474190d3d2 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:02.374677 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-696c92f7-27d4-4c26-ba34-00474190d3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:02.395250 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:02.403814 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d998459c-8ee4-4ede-9b29-2d14c2bd7cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:02.404623 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d998459c-8ee4-4ede-9b29-2d14c2bd7cc1 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:02.404746 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d998459c-8ee4-4ede-9b29-2d14c2bd7cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:02.404976 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d998459c-8ee4-4ede-9b29-2d14c2bd7cc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:02.405712 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d998459c-8ee4-4ede-9b29-2d14c2bd7cc1 None admin] RBAC: Adding query filter params (role.id=1316db16eac54629a283545f1601a236, scope.project.id=01cf71ca5abf4576b89d10c85af6d2de, user.id=6fda7075198d48da86aa9c0dab6ff7ca) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:02.406020 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d998459c-8ee4-4ede-9b29-2d14c2bd7cc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:02.407484 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d998459c-8ee4-4ede-9b29-2d14c2bd7cc1 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:02.409393 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:02.418769 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 344/685] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:30:02 2022] GET /identity/v3/role_assignments?user.id=6fda7075198d48da86aa9c0dab6ff7ca&scope.project.id=01cf71ca5abf4576b89d10c85af6d2de&role.id=1316db16eac54629a283545f1601a236 => generated 608 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:02.418769 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d998459c-8ee4-4ede-9b29-2d14c2bd7cc1 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:02.470948 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 342/686] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:30:02.561576 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d998459c-8ee4-4ede-9b29-2d14c2bd7cc1 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:02.607854 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d15acf-5c47-4369-a8ce-116828b54830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:02.608479 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d15acf-5c47-4369-a8ce-116828b54830 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:02.608754 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d15acf-5c47-4369-a8ce-116828b54830 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:02.609023 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d15acf-5c47-4369-a8ce-116828b54830 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:02.693908 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:02.698753 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d15acf-5c47-4369-a8ce-116828b54830 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:02.700543 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d15acf-5c47-4369-a8ce-116828b54830 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:02.709806 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8cc96a-212b-4f1c-b5fc-8d67f1ee3f97 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:02.710087 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8cc96a-212b-4f1c-b5fc-8d67f1ee3f97 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:02.710352 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8cc96a-212b-4f1c-b5fc-8d67f1ee3f97 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:02.711713 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 343/687] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:30:02.717211 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02013697-fd8c-4e53-8d15-d69faf9d7798 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:02.717495 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02013697-fd8c-4e53-8d15-d69faf9d7798 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:02.717767 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02013697-fd8c-4e53-8d15-d69faf9d7798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:02.740740 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-02013697-fd8c-4e53-8d15-d69faf9d7798 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:02.741844 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-02013697-fd8c-4e53-8d15-d69faf9d7798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:02.821598 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 345/688] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:30:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:30:02.827684 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d15acf-5c47-4369-a8ce-116828b54830 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:02.840848 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 344/689] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 14:30:02.852552 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02013697-fd8c-4e53-8d15-d69faf9d7798 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:02.945041 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:02.966592 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02013697-fd8c-4e53-8d15-d69faf9d7798 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:02.983769 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:02.988899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d15acf-5c47-4369-a8ce-116828b54830 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:03.004933 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26abc1cf-effe-4679-9c26-55fd6b1ed920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:03.005616 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26abc1cf-effe-4679-9c26-55fd6b1ed920 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:03.006006 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26abc1cf-effe-4679-9c26-55fd6b1ed920 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:03.006369 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26abc1cf-effe-4679-9c26-55fd6b1ed920 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:03.018451 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26abc1cf-effe-4679-9c26-55fd6b1ed920 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:03.023869 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-26abc1cf-effe-4679-9c26-55fd6b1ed920 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:30:03.023869 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:03.023869 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:03.023869 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:03.023869 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:03.023869 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:03.023869 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:03.023869 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:03.023869 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:03.023869 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:03.023869 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:03.023869 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:03.023869 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:03.023869 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:03.023869 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:03.023869 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:03.024835 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:03.024835 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:03.024835 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:03.024835 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:03.024835 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:03.024835 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:03.024835 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:03.024835 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:03.024835 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:03.024835 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:03.024835 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:03.024835 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:03.024835 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:03.024835 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:03.024835 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:03.024835 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:03.025670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:03.025670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:03.025670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:03.025670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:03.025670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:03.025670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:03.025670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:03.025670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:03.025670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:03.025670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:03.025670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:30:03.025670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:03.030146 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 345/690] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 30 14:30:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:30:03.030807 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405cb049-5d77-4b58-b865-538da330d476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:03.031637 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26abc1cf-effe-4679-9c26-55fd6b1ed920 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:03.031951 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405cb049-5d77-4b58-b865-538da330d476 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:03.032415 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405cb049-5d77-4b58-b865-538da330d476 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:03.032871 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405cb049-5d77-4b58-b865-538da330d476 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:03.045355 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405cb049-5d77-4b58-b865-538da330d476 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:03.047105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405cb049-5d77-4b58-b865-538da330d476 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:03.064450 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-405cb049-5d77-4b58-b865-538da330d476 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:03.064450 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:03.064450 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:03.064450 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:03.064450 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:03.064450 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:03.064450 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:03.064450 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:03.064450 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:03.064450 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:03.064450 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:03.064450 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:03.064450 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:30:03.064450 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:30:03.064450 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:30:03.064450 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:30:03.065437 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:03.065437 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:03.065437 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:03.065437 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:03.065437 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:03.065437 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:03.065437 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:03.065437 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:03.065437 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:03.065437 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:03.065437 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:03.065437 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:03.065437 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:03.065437 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:03.065437 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:03.065437 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:03.065437 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:30:03.066363 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:30:03.066363 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:03.066363 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:03.068592 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 346/691] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:03.074068 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405cb049-5d77-4b58-b865-538da330d476 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:03.141622 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26abc1cf-effe-4679-9c26-55fd6b1ed920 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:03.179749 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e185eaba-2343-488c-8c45-71a2b1cb54b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:03.180364 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e185eaba-2343-488c-8c45-71a2b1cb54b6 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:03.180646 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e185eaba-2343-488c-8c45-71a2b1cb54b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:03.180986 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e185eaba-2343-488c-8c45-71a2b1cb54b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:03.181580 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e185eaba-2343-488c-8c45-71a2b1cb54b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:03.181894 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e185eaba-2343-488c-8c45-71a2b1cb54b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:03.184183 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e185eaba-2343-488c-8c45-71a2b1cb54b6 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:03.188239 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405cb049-5d77-4b58-b865-538da330d476 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:03.197683 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 346/692] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:30:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:03.207234 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e185eaba-2343-488c-8c45-71a2b1cb54b6 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:03.208947 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:03.264911 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aca36d2-ed67-4d7a-815a-2dfba6b05b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:03.265868 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca36d2-ed67-4d7a-815a-2dfba6b05b78 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:03.266370 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca36d2-ed67-4d7a-815a-2dfba6b05b78 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:03.266852 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca36d2-ed67-4d7a-815a-2dfba6b05b78 None admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:03.267721 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aca36d2-ed67-4d7a-815a-2dfba6b05b78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:03.268220 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aca36d2-ed67-4d7a-815a-2dfba6b05b78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:03.269774 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aca36d2-ed67-4d7a-815a-2dfba6b05b78 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:03.297338 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 347/693] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:30:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:03.298505 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aca36d2-ed67-4d7a-815a-2dfba6b05b78 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:03.311516 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e185eaba-2343-488c-8c45-71a2b1cb54b6 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:03.317747 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:03.351706 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a89720e-5898-4718-86f7-4549a3d51566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:03.352384 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a89720e-5898-4718-86f7-4549a3d51566 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:03.352664 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a89720e-5898-4718-86f7-4549a3d51566 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:03.352938 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a89720e-5898-4718-86f7-4549a3d51566 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:03.364858 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a89720e-5898-4718-86f7-4549a3d51566 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:03.366320 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a89720e-5898-4718-86f7-4549a3d51566 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:03.382799 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-8a89720e-5898-4718-86f7-4549a3d51566 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:03.382799 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:03.382799 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:03.382799 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:03.382799 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:03.382799 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:03.382799 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:03.382799 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:03.382799 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:03.382799 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:03.382799 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:03.382799 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:03.382799 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:30:03.382799 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:30:03.382799 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:30:03.382799 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:30:03.384007 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:03.384007 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:03.384007 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:03.384007 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:03.384007 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:03.384007 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:03.384007 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:03.384007 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:03.384007 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:03.384007 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:03.384007 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:03.384007 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:03.384007 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:03.384007 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:03.384007 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:03.384007 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:03.384007 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:30:03.384990 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:30:03.384990 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:03.384990 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:03.389418 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 347/694] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:03.390319 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a89720e-5898-4718-86f7-4549a3d51566 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:03.419744 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aca36d2-ed67-4d7a-815a-2dfba6b05b78 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:03.454462 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:03.461245 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0d1510-368a-40f3-affa-fb8e86481a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:03.461959 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d1510-368a-40f3-affa-fb8e86481a62 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:03.462298 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d1510-368a-40f3-affa-fb8e86481a62 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:03.462567 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d1510-368a-40f3-affa-fb8e86481a62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:03.474377 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0d1510-368a-40f3-affa-fb8e86481a62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:03.476000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0d1510-368a-40f3-affa-fb8e86481a62 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:03.492681 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-bd0d1510-368a-40f3-affa-fb8e86481a62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:03.492681 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:03.492681 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:03.492681 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:03.492681 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:03.492681 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:03.492681 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:03.492681 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:03.492681 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:03.492681 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:03.492681 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:03.492681 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:03.492681 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:30:03.492681 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:30:03.492681 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:30:03.492681 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:30:03.494795 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:03.494795 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:03.494795 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:03.494795 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:03.494795 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:03.494795 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:03.494795 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:03.494795 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:03.494795 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:03.494795 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:03.494795 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:03.494795 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:03.494795 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:03.494795 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:03.494795 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:03.494795 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:03.494795 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:30:03.497779 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:30:03.497779 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:03.497779 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:03.507227 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 348/695] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:03.507227 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0d1510-368a-40f3-affa-fb8e86481a62 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:03.509171 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a89720e-5898-4718-86f7-4549a3d51566 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:03.547525 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca603cd6-cd9b-42a2-b0df-6ee9db721c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:03.548112 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca603cd6-cd9b-42a2-b0df-6ee9db721c22 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:03.548446 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca603cd6-cd9b-42a2-b0df-6ee9db721c22 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:03.548726 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca603cd6-cd9b-42a2-b0df-6ee9db721c22 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:03.556623 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca603cd6-cd9b-42a2-b0df-6ee9db721c22 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:03.559077 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca603cd6-cd9b-42a2-b0df-6ee9db721c22 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:03.567663 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:03.573652 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-ca603cd6-cd9b-42a2-b0df-6ee9db721c22 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:30:03.573652 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:03.573652 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:03.573652 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:03.573652 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:03.573652 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:03.573652 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:03.573652 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:03.573652 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:03.573652 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:03.573652 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:03.573652 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:03.573652 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:30:03.573652 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:30:03.573652 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:30:03.575680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:30:03.575680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:03.575680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:03.575680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:03.575680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:03.575680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:03.575680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:03.575680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:03.575680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:03.575680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:03.575680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:03.575680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:03.575680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:03.575680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:03.575680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:03.575680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:03.575680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:03.577861 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:30:03.577861 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:30:03.577861 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:30:03.577861 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:30:03.577861 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:30:03.577861 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:30:03.577861 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:30:03.577861 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:03.577861 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 348/696] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:30:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:03.577861 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca603cd6-cd9b-42a2-b0df-6ee9db721c22 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:03.615858 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0d1510-368a-40f3-affa-fb8e86481a62 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:03.653153 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058134da-97f1-4f78-998e-03f4dc891c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:03.653824 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058134da-97f1-4f78-998e-03f4dc891c7b None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:03.654100 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058134da-97f1-4f78-998e-03f4dc891c7b None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:03.654366 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058134da-97f1-4f78-998e-03f4dc891c7b None admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:03.655062 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058134da-97f1-4f78-998e-03f4dc891c7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:03.655438 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058134da-97f1-4f78-998e-03f4dc891c7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:03.656808 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058134da-97f1-4f78-998e-03f4dc891c7b None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:03.670595 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 349/697] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:30:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:03.671457 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058134da-97f1-4f78-998e-03f4dc891c7b None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:03.690953 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca603cd6-cd9b-42a2-b0df-6ee9db721c22 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:03.708965 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:03.730471 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6c6e6d-26b8-437a-8c77-c4165371b504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:03.731084 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6c6e6d-26b8-437a-8c77-c4165371b504 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:03.731398 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6c6e6d-26b8-437a-8c77-c4165371b504 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:03.731674 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6c6e6d-26b8-437a-8c77-c4165371b504 None admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:03.732422 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6c6e6d-26b8-437a-8c77-c4165371b504 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:03.732728 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6c6e6d-26b8-437a-8c77-c4165371b504 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:03.734100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6c6e6d-26b8-437a-8c77-c4165371b504 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:03.749357 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 349/698] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:30:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:03.750301 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6c6e6d-26b8-437a-8c77-c4165371b504 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:03.788378 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058134da-97f1-4f78-998e-03f4dc891c7b None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:03.815375 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:03.829655 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e8ef2-a2a1-4b68-a7ed-f2fca7ae6864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:03.830260 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e8ef2-a2a1-4b68-a7ed-f2fca7ae6864 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:03.830534 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e8ef2-a2a1-4b68-a7ed-f2fca7ae6864 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:03.830805 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e8ef2-a2a1-4b68-a7ed-f2fca7ae6864 None admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:03.831543 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7e8ef2-a2a1-4b68-a7ed-f2fca7ae6864 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:03.831843 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7e8ef2-a2a1-4b68-a7ed-f2fca7ae6864 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:03.833372 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7e8ef2-a2a1-4b68-a7ed-f2fca7ae6864 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:03.852028 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 350/699] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 30 14:30:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:03.852998 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e8ef2-a2a1-4b68-a7ed-f2fca7ae6864 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:03.863402 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6c6e6d-26b8-437a-8c77-c4165371b504 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:03.900869 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967949b4-854f-483a-ad62-b42045f039f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:03.902049 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967949b4-854f-483a-ad62-b42045f039f6 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:03.902327 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967949b4-854f-483a-ad62-b42045f039f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:03.902600 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967949b4-854f-483a-ad62-b42045f039f6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:03.910591 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967949b4-854f-483a-ad62-b42045f039f6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:03.912195 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967949b4-854f-483a-ad62-b42045f039f6 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:03.925742 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-967949b4-854f-483a-ad62-b42045f039f6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:30:03.925742 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:03.925742 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:03.925742 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:03.925742 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:03.925742 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:03.925742 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:03.925742 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:03.925742 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:03.925742 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:03.925742 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:03.925742 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:03.925742 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:30:03.925742 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:30:03.925742 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:30:03.925742 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:30:03.927901 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:03.927901 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:03.927901 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:30:03.927901 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:30:03.927901 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:30:03.927901 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:30:03.927901 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:03.927901 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:03.927901 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:03.927901 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:03.927901 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:03.927901 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:03.927901 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:03.927901 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:03.927901 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:03.927901 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:03.927901 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:03.927901 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:03.930088 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:03.930088 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:03.930088 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:30:03.930088 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:30:03.930088 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:30:03.930088 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:30:03.930088 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:30:03.930088 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:30:03.930088 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:30:03.930088 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:03.930088 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 350/700] 10.222.0.30 () {56 vars in 1093 bytes} [Tue Aug 30 14:30:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:03.935336 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967949b4-854f-483a-ad62-b42045f039f6 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:03.964328 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e8ef2-a2a1-4b68-a7ed-f2fca7ae6864 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:03.973865 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:04.004504 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087822cc-a020-4909-afe8-558de5154454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:04.005837 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087822cc-a020-4909-afe8-558de5154454 None admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:04.006262 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087822cc-a020-4909-afe8-558de5154454 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:04.006699 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087822cc-a020-4909-afe8-558de5154454 None admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:04.007580 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087822cc-a020-4909-afe8-558de5154454 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:04.009320 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087822cc-a020-4909-afe8-558de5154454 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:04.039070 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 351/701] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 30 14:30:03 2022] POST /identity/v3/users => generated 254 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:30:04.040167 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087822cc-a020-4909-afe8-558de5154454 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:04.055542 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967949b4-854f-483a-ad62-b42045f039f6 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:04.088115 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:04.100145 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f46bb-5dcb-4ace-8a6d-5ff191ef358d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:04.101836 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f46bb-5dcb-4ace-8a6d-5ff191ef358d None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:04.102300 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f46bb-5dcb-4ace-8a6d-5ff191ef358d None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:04.102720 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f46bb-5dcb-4ace-8a6d-5ff191ef358d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:04.103719 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865f46bb-5dcb-4ace-8a6d-5ff191ef358d None admin] RBAC: Adding query filter params (role.id=1316db16eac54629a283545f1601a236, scope.project.id=01cf71ca5abf4576b89d10c85af6d2de, user.id=a18b13e3708144e1b5e8df80ec3a4e83) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:04.104918 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865f46bb-5dcb-4ace-8a6d-5ff191ef358d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:04.106880 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865f46bb-5dcb-4ace-8a6d-5ff191ef358d None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:04.116959 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 351/702] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:30:03 2022] GET /identity/v3/role_assignments?user.id=a18b13e3708144e1b5e8df80ec3a4e83&scope.project.id=01cf71ca5abf4576b89d10c85af6d2de&role.id=1316db16eac54629a283545f1601a236 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:04.159639 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087822cc-a020-4909-afe8-558de5154454 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:04.199490 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e67a7e5-09f8-4e6e-809d-35f772bff1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:04.200230 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e67a7e5-09f8-4e6e-809d-35f772bff1c5 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:04.200524 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e67a7e5-09f8-4e6e-809d-35f772bff1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:04.200799 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e67a7e5-09f8-4e6e-809d-35f772bff1c5 None admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:04.201573 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e67a7e5-09f8-4e6e-809d-35f772bff1c5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:04.201886 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e67a7e5-09f8-4e6e-809d-35f772bff1c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:04.203410 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e67a7e5-09f8-4e6e-809d-35f772bff1c5 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:04.225075 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 352/703] 10.222.0.30 () {56 vars in 1133 bytes} [Tue Aug 30 14:30:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:04.231242 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f46bb-5dcb-4ace-8a6d-5ff191ef358d None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:04.381633 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:04.382889 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970008d4-b549-4f37-857e-4bd0d91b044f None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:04.383192 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970008d4-b549-4f37-857e-4bd0d91b044f None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:04.383528 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970008d4-b549-4f37-857e-4bd0d91b044f None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:04.385123 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 353/704] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:04.385331 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f46bb-5dcb-4ace-8a6d-5ff191ef358d None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:04.391729 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f06881-b222-429e-a8cf-12a09f10cb46 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:04.392013 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f06881-b222-429e-a8cf-12a09f10cb46 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:04.392268 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f06881-b222-429e-a8cf-12a09f10cb46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:04.418736 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-a8f06881-b222-429e-a8cf-12a09f10cb46 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:04.420335 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-a8f06881-b222-429e-a8cf-12a09f10cb46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:04.431555 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c172cfe-a290-447d-9eb3-5d293bcb5af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:04.432204 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c172cfe-a290-447d-9eb3-5d293bcb5af1 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:04.432901 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c172cfe-a290-447d-9eb3-5d293bcb5af1 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:04.433192 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c172cfe-a290-447d-9eb3-5d293bcb5af1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:04.510169 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 354/705] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:30:04.530926 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c172cfe-a290-447d-9eb3-5d293bcb5af1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:04.532758 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c172cfe-a290-447d-9eb3-5d293bcb5af1 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:04.640588 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:04.650330 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 352/706] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:30:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:30:04.652318 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f06881-b222-429e-a8cf-12a09f10cb46 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:04.680408 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:04.762633 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f06881-b222-429e-a8cf-12a09f10cb46 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:04.840005 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d5d2d7-be57-4ff5-bb57-4ff8bf8c38e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:04.840897 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d5d2d7-be57-4ff5-bb57-4ff8bf8c38e0 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:04.841309 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d5d2d7-be57-4ff5-bb57-4ff8bf8c38e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:04.841738 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d5d2d7-be57-4ff5-bb57-4ff8bf8c38e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:04.857066 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d5d2d7-be57-4ff5-bb57-4ff8bf8c38e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:04.858901 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d5d2d7-be57-4ff5-bb57-4ff8bf8c38e0 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:04.873723 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c172cfe-a290-447d-9eb3-5d293bcb5af1 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:04.899301 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-79d5d2d7-be57-4ff5-bb57-4ff8bf8c38e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:04.899301 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:04.899301 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:04.899301 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:04.899301 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:04.899301 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:04.899301 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:04.899301 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:04.899301 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:04.899301 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:04.899301 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:04.899301 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:04.899301 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:30:04.899301 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:30:04.899301 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:30:04.899301 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:30:04.900310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:04.900310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:04.900310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:04.900310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:04.900310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:04.900310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:04.900310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:04.900310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:04.900310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:04.900310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:04.900310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:04.900310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:04.900310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:04.900310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:04.900310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:04.900310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:04.900310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:30:04.901268 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:30:04.901268 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:04.901268 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:04.905931 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 355/707] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:04.910655 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d5d2d7-be57-4ff5-bb57-4ff8bf8c38e0 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:05.032271 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c172cfe-a290-447d-9eb3-5d293bcb5af1 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:05.045060 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:05.073890 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d5d2d7-be57-4ff5-bb57-4ff8bf8c38e0 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:05.083462 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f3246d-852a-48f6-84f1-2274b139376e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:05.084161 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f3246d-852a-48f6-84f1-2274b139376e None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:05.084448 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f3246d-852a-48f6-84f1-2274b139376e None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:05.084983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f3246d-852a-48f6-84f1-2274b139376e None admin] PATH_INFO: `/v3/services/network` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:05.085671 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f3246d-852a-48f6-84f1-2274b139376e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:05.089415 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f3246d-852a-48f6-84f1-2274b139376e None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:05.106156 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-a2f3246d-852a-48f6-84f1-2274b139376e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 14:30:05.106156 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:05.106156 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:30:05.106156 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:30:05.106156 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:30:05.106156 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:30:05.106156 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:30:05.106156 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:30:05.106156 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 14:30:05.106156 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:05.106156 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:30:05.106156 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:05.106156 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:05.106156 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:05.106156 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:05.106156 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:05.106156 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:05.107535 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:05.107535 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:05.107535 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:05.107535 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:05.107535 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:05.107535 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:05.107535 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:30:05.107535 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:30:05.107535 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:30:05.107535 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:30:05.107535 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:05.107535 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:05.107535 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:05.107535 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:05.107535 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:05.107535 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:05.108403 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:05.108403 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:05.108403 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:05.108403 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:05.108403 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:05.108403 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:05.108403 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:05.108403 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:05.108403 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:05.108403 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:05.108403 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:30:05.108403 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:30:05.108403 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 14:30:05.108403 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:05.113139 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 353/708] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:30:04 2022] GET /identity/v3/services/network => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:30:05.115242 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a86e42-853e-4844-a633-245f9fdd8d16 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:05.115466 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a86e42-853e-4844-a633-245f9fdd8d16 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:05.115842 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a86e42-853e-4844-a633-245f9fdd8d16 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:05.119257 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 354/709] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:05.119257 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a86e42-853e-4844-a633-245f9fdd8d16 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:05.128909 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eba8c8-8ae5-4162-bd3b-f3950a0f1bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:05.130669 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eba8c8-8ae5-4162-bd3b-f3950a0f1bfe None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:05.131044 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eba8c8-8ae5-4162-bd3b-f3950a0f1bfe None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:05.131498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eba8c8-8ae5-4162-bd3b-f3950a0f1bfe None admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:05.132249 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eba8c8-8ae5-4162-bd3b-f3950a0f1bfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:05.132647 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eba8c8-8ae5-4162-bd3b-f3950a0f1bfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:05.134282 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eba8c8-8ae5-4162-bd3b-f3950a0f1bfe None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:05.154263 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 356/710] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:30:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:05.156032 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e890a-3121-4063-95da-57b8380a3c97 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:05.156413 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e890a-3121-4063-95da-57b8380a3c97 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:05.157106 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e890a-3121-4063-95da-57b8380a3c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:05.179244 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-f99e890a-3121-4063-95da-57b8380a3c97 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:05.180982 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-f99e890a-3121-4063-95da-57b8380a3c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:05.241122 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:05.273900 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 357/711] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:30:05.275087 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99e890a-3121-4063-95da-57b8380a3c97 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:05.344812 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a86e42-853e-4844-a633-245f9fdd8d16 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:05.413535 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:05.416297 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14215bae-0163-4c14-8cbc-c2f5b258d693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:05.418091 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14215bae-0163-4c14-8cbc-c2f5b258d693 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:05.420011 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14215bae-0163-4c14-8cbc-c2f5b258d693 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:05.420536 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14215bae-0163-4c14-8cbc-c2f5b258d693 None admin] PATH_INFO: `/v3/services` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:05.422134 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14215bae-0163-4c14-8cbc-c2f5b258d693 None admin] RBAC: Adding query filter params (name=network) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:05.422647 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14215bae-0163-4c14-8cbc-c2f5b258d693 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:05.424368 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14215bae-0163-4c14-8cbc-c2f5b258d693 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:05.434822 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 355/712] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Aug 30 14:30:05 2022] GET /identity/v3/services?name=network => generated 124 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:05.442720 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14215bae-0163-4c14-8cbc-c2f5b258d693 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:05.490106 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99e890a-3121-4063-95da-57b8380a3c97 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:05.571122 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f1c51a-fe4a-4227-b8ac-4498a1f4c24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:05.572578 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f1c51a-fe4a-4227-b8ac-4498a1f4c24e None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:05.573686 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f1c51a-fe4a-4227-b8ac-4498a1f4c24e None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:05.574182 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f1c51a-fe4a-4227-b8ac-4498a1f4c24e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:05.582529 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f1c51a-fe4a-4227-b8ac-4498a1f4c24e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:05.584261 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f1c51a-fe4a-4227-b8ac-4498a1f4c24e None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:05.597390 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-e2f1c51a-fe4a-4227-b8ac-4498a1f4c24e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:30:05.597390 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:05.597390 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:05.597390 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:05.597390 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:05.597390 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:05.597390 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:05.597390 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:05.597390 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:05.597390 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:05.597390 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:05.597390 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:05.597390 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:30:05.597390 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:30:05.597390 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:30:05.598312 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:30:05.598312 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:05.598312 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:05.598312 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:05.598312 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:05.598312 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:05.598312 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:05.598312 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:05.598312 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:05.598312 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:05.598312 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:05.598312 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:05.598312 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:05.598312 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:05.598312 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:05.598312 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:05.598312 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:05.599345 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:30:05.599345 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:30:05.599345 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:30:05.599345 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:30:05.599345 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:30:05.599345 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:30:05.599345 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:30:05.599345 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:05.602622 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 358/713] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:30:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:05.603927 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f1c51a-fe4a-4227-b8ac-4498a1f4c24e None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:05.644803 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14215bae-0163-4c14-8cbc-c2f5b258d693 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:05.677733 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:05.688780 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee90b98-83a4-4944-a41d-31b46339a6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:05.689387 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee90b98-83a4-4944-a41d-31b46339a6d8 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:05.689665 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee90b98-83a4-4944-a41d-31b46339a6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:05.689934 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee90b98-83a4-4944-a41d-31b46339a6d8 None admin] PATH_INFO: `/v3/services` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:05.690721 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee90b98-83a4-4944-a41d-31b46339a6d8 None admin] RBAC: Adding query filter params (type=network) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:05.691021 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee90b98-83a4-4944-a41d-31b46339a6d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:05.692936 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee90b98-83a4-4944-a41d-31b46339a6d8 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:05.707238 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 356/714] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Aug 30 14:30:05 2022] GET /identity/v3/services?type=network => generated 124 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:30:05.707238 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee90b98-83a4-4944-a41d-31b46339a6d8 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:05.769753 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f1c51a-fe4a-4227-b8ac-4498a1f4c24e None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:05.830985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:05.847320 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a8c68-e8b0-4f3f-80d7-6b3d13b0548a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:05.848192 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a8c68-e8b0-4f3f-80d7-6b3d13b0548a None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:05.848465 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a8c68-e8b0-4f3f-80d7-6b3d13b0548a None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:05.851349 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a8c68-e8b0-4f3f-80d7-6b3d13b0548a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:05.851980 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86a8c68-e8b0-4f3f-80d7-6b3d13b0548a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:05.853381 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86a8c68-e8b0-4f3f-80d7-6b3d13b0548a None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:05.872434 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-f86a8c68-e8b0-4f3f-80d7-6b3d13b0548a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 14:30:05.872434 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:05.872434 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:30:05.872434 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:30:05.872434 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:30:05.872434 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:30:05.872434 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:30:05.872434 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:30:05.872434 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 14:30:05.872434 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:05.872434 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:30:05.872434 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:05.872434 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:05.872434 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:05.872434 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:05.872434 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:05.872434 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:05.873551 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:05.873551 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:05.873551 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:05.873551 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:05.873551 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:05.873551 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:05.873551 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:30:05.873551 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:30:05.873551 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:30:05.873551 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:30:05.873551 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:05.873551 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:05.873551 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:05.873551 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:05.873551 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:05.873551 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:05.874436 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:05.874436 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:05.874436 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:05.874436 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:05.874436 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:05.874436 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:05.874436 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:05.874436 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:05.874436 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:05.874436 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:05.874436 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:30:05.874436 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:30:05.874436 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 14:30:05.874436 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:05.874436 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 359/715] 10.222.0.30 () {56 vars in 1129 bytes} [Tue Aug 30 14:30:05 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:05.880379 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a8c68-e8b0-4f3f-80d7-6b3d13b0548a None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:05.914067 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee90b98-83a4-4944-a41d-31b46339a6d8 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:05.954100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cae41f4-af19-4137-897c-f5670331ec42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:05.955068 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae41f4-af19-4137-897c-f5670331ec42 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:05.955601 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae41f4-af19-4137-897c-f5670331ec42 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:05.956077 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae41f4-af19-4137-897c-f5670331ec42 None admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:05.956992 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cae41f4-af19-4137-897c-f5670331ec42 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:05.957514 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cae41f4-af19-4137-897c-f5670331ec42 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:05.959176 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cae41f4-af19-4137-897c-f5670331ec42 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:05.977778 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 357/716] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 30 14:30:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:05.984317 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cae41f4-af19-4137-897c-f5670331ec42 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:05.993851 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a8c68-e8b0-4f3f-80d7-6b3d13b0548a None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:06.030859 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e831e8-9943-4f34-be45-59fe402e4e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:06.031469 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e831e8-9943-4f34-be45-59fe402e4e9e None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:06.031980 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e831e8-9943-4f34-be45-59fe402e4e9e None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:06.032353 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e831e8-9943-4f34-be45-59fe402e4e9e None admin] PATH_INFO: `/v3/services` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:06.033395 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e831e8-9943-4f34-be45-59fe402e4e9e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:06.033395 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e831e8-9943-4f34-be45-59fe402e4e9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:06.034949 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e831e8-9943-4f34-be45-59fe402e4e9e None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:06.039846 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 360/717] 10.222.0.30 () {56 vars in 1124 bytes} [Tue Aug 30 14:30:05 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:06.045571 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e831e8-9943-4f34-be45-59fe402e4e9e None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:06.076552 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:06.103982 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:06.150430 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e831e8-9943-4f34-be45-59fe402e4e9e None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:06.187371 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078c025e-d32f-4142-afbc-5991ba6fe048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:06.187993 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c025e-d32f-4142-afbc-5991ba6fe048 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:06.188270 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c025e-d32f-4142-afbc-5991ba6fe048 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:06.188542 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c025e-d32f-4142-afbc-5991ba6fe048 None admin] PATH_INFO: `/v3/services` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:06.189351 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078c025e-d32f-4142-afbc-5991ba6fe048 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:06.189659 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078c025e-d32f-4142-afbc-5991ba6fe048 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:06.191131 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078c025e-d32f-4142-afbc-5991ba6fe048 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:06.197552 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 361/718] 10.222.0.30 () {56 vars in 1124 bytes} [Tue Aug 30 14:30:06 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:30:06.243164 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cae41f4-af19-4137-897c-f5670331ec42 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:06.299620 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af04219-b4c0-458d-b265-f9448464fa5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:06.300730 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af04219-b4c0-458d-b265-f9448464fa5f None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:06.301214 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af04219-b4c0-458d-b265-f9448464fa5f None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:06.301687 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af04219-b4c0-458d-b265-f9448464fa5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:06.302654 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af04219-b4c0-458d-b265-f9448464fa5f None admin] RBAC: Adding query filter params (role.id=3cda0615d87d4af0860a2a5e9b4e2c05, scope.project.id=01cf71ca5abf4576b89d10c85af6d2de, user.id=8878de66a03044e38e507028853a0cd1) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:06.303177 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af04219-b4c0-458d-b265-f9448464fa5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:06.305113 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af04219-b4c0-458d-b265-f9448464fa5f None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:06.312203 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 358/719] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:30:05 2022] GET /identity/v3/role_assignments?user.id=8878de66a03044e38e507028853a0cd1&scope.project.id=01cf71ca5abf4576b89d10c85af6d2de&role.id=3cda0615d87d4af0860a2a5e9b4e2c05 => generated 259 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:06.437833 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4997cf-a34e-43c0-b59f-66d63e04725e None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:06.438209 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4997cf-a34e-43c0-b59f-66d63e04725e None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:06.438402 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4997cf-a34e-43c0-b59f-66d63e04725e None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:06.439773 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 362/720] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:06.445372 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aecc6c-1470-44e9-9b76-8ba56184926f None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:06.445981 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aecc6c-1470-44e9-9b76-8ba56184926f None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:06.446478 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aecc6c-1470-44e9-9b76-8ba56184926f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:06.471314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-f0aecc6c-1470-44e9-9b76-8ba56184926f None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:06.472866 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-f0aecc6c-1470-44e9-9b76-8ba56184926f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:06.555509 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 359/721] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:30:06.859011 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4997cf-a34e-43c0-b59f-66d63e04725e None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:06.893762 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7480417f-a63b-43e1-ba80-494af520a9c8 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:06.894099 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7480417f-a63b-43e1-ba80-494af520a9c8 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:06.894319 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7480417f-a63b-43e1-ba80-494af520a9c8 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:06.895960 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 360/722] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:06.900908 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f4cc9-8e24-4343-9a4f-66cfc5a70145 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:06.901762 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f4cc9-8e24-4343-9a4f-66cfc5a70145 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:06.902175 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f4cc9-8e24-4343-9a4f-66cfc5a70145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:06.923356 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-d11f4cc9-8e24-4343-9a4f-66cfc5a70145 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:06.924543 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-d11f4cc9-8e24-4343-9a4f-66cfc5a70145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:06.987799 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4997cf-a34e-43c0-b59f-66d63e04725e None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:07.011627 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:07.025248 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab02f8-6a67-47b8-bb5e-ed455096586d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:07.026871 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab02f8-6a67-47b8-bb5e-ed455096586d None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:07.027162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab02f8-6a67-47b8-bb5e-ed455096586d None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:07.027462 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab02f8-6a67-47b8-bb5e-ed455096586d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:07.038888 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ab02f8-6a67-47b8-bb5e-ed455096586d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:07.044257 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-a7ab02f8-6a67-47b8-bb5e-ed455096586d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:30:07.044257 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:07.044257 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:07.044257 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:07.044257 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:07.044257 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:07.044257 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:07.044257 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:07.044257 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:07.044257 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:07.044257 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:07.044257 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:07.044257 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:07.044257 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:07.044257 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:07.044257 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:07.045234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:07.045234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:07.045234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:07.045234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:07.045234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:07.045234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:07.045234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:07.045234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:07.045234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:07.045234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:07.045234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:07.045234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:07.045234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:07.045234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:07.045234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:07.045234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:07.046536 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:07.046536 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:07.046536 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:07.046536 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:07.046536 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:07.046536 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:07.046536 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:07.046536 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:07.046536 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:07.046536 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:07.046536 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:30:07.046536 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:07.046536 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 363/723] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 30 14:30:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:07.051467 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab02f8-6a67-47b8-bb5e-ed455096586d None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:07.070901 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:07.149110 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 361/724] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:30:07.193965 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab02f8-6a67-47b8-bb5e-ed455096586d None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:07.250712 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df90794-583b-4fb2-a330-fdb088c52bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:07.251696 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df90794-583b-4fb2-a330-fdb088c52bf2 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:07.252207 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df90794-583b-4fb2-a330-fdb088c52bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:07.252694 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df90794-583b-4fb2-a330-fdb088c52bf2 None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:07.253582 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df90794-583b-4fb2-a330-fdb088c52bf2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:07.254087 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df90794-583b-4fb2-a330-fdb088c52bf2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:07.255721 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df90794-583b-4fb2-a330-fdb088c52bf2 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:07.267846 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 364/725] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 30 14:30:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:07.273021 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11f4cc9-8e24-4343-9a4f-66cfc5a70145 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:07.403149 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11f4cc9-8e24-4343-9a4f-66cfc5a70145 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:07.439326 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d7aa7a-809c-4b87-8854-0aca1e7305ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:07.439962 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7aa7a-809c-4b87-8854-0aca1e7305ed None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:07.440242 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7aa7a-809c-4b87-8854-0aca1e7305ed None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:07.440511 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7aa7a-809c-4b87-8854-0aca1e7305ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:07.451294 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d7aa7a-809c-4b87-8854-0aca1e7305ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:07.452729 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d7aa7a-809c-4b87-8854-0aca1e7305ed None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:07.467811 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-07d7aa7a-809c-4b87-8854-0aca1e7305ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:07.467811 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:07.467811 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:07.467811 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:07.467811 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:07.467811 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:07.467811 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:07.467811 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:07.467811 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:07.467811 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:07.467811 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:07.467811 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:07.467811 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:30:07.467811 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:30:07.467811 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:30:07.467811 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:30:07.468807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:07.468807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:07.468807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:07.468807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:07.468807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:07.468807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:07.468807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:07.468807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:07.468807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:07.468807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:07.468807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:07.468807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:07.468807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:07.468807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:07.468807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:07.468807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:07.468807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:30:07.469702 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:30:07.469702 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:07.469702 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:07.469702 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 362/726] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:07.474478 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df90794-583b-4fb2-a330-fdb088c52bf2 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:07.532981 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d7aa7a-809c-4b87-8854-0aca1e7305ed None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:07.551569 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:07.567596 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:07.648507 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d7aa7a-809c-4b87-8854-0aca1e7305ed None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:07.687531 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3753656a-080b-4234-8483-a1747d611375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:07.688085 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753656a-080b-4234-8483-a1747d611375 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:07.688363 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753656a-080b-4234-8483-a1747d611375 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:07.688788 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753656a-080b-4234-8483-a1747d611375 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:07.700210 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3753656a-080b-4234-8483-a1747d611375 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:07.705541 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-3753656a-080b-4234-8483-a1747d611375 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:30:07.705541 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:07.705541 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:07.705541 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:07.705541 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:07.705541 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:07.705541 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:07.705541 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:07.705541 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:07.705541 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:07.705541 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:07.705541 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:07.705541 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:07.705541 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:07.705541 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:07.705541 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:07.706476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:07.706476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:07.706476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:07.706476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:07.706476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:07.706476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:07.706476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:07.706476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:07.706476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:07.706476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:07.706476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:07.706476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:07.706476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:07.706476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:07.706476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:07.706476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:07.707346 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:07.707346 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:07.707346 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:07.707346 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:07.707346 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:07.707346 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:07.707346 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:07.707346 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:07.707346 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:07.707346 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:07.707346 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:30:07.707346 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:07.707999 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 363/727] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 30 14:30:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:30:07.716159 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3753656a-080b-4234-8483-a1747d611375 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:07.718593 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df90794-583b-4fb2-a330-fdb088c52bf2 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:07.762514 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914b6a11-b074-428d-88a3-69394297063c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:07.763243 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b6a11-b074-428d-88a3-69394297063c None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:07.763596 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b6a11-b074-428d-88a3-69394297063c None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:07.763951 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b6a11-b074-428d-88a3-69394297063c None admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:07.765718 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914b6a11-b074-428d-88a3-69394297063c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:07.766083 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914b6a11-b074-428d-88a3-69394297063c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:07.768266 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914b6a11-b074-428d-88a3-69394297063c None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:07.798635 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 365/728] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:30:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:07.799571 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914b6a11-b074-428d-88a3-69394297063c None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:07.891579 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:07.914624 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3753656a-080b-4234-8483-a1747d611375 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:07.977339 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914b6a11-b074-428d-88a3-69394297063c None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:07.988345 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a4cc4c-a4f0-4d2f-bdd0-34dec2571237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:07.989021 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a4cc4c-a4f0-4d2f-bdd0-34dec2571237 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:07.989308 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a4cc4c-a4f0-4d2f-bdd0-34dec2571237 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:07.989611 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:07.989717 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a4cc4c-a4f0-4d2f-bdd0-34dec2571237 None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:07.990297 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a4cc4c-a4f0-4d2f-bdd0-34dec2571237 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:07.990602 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a4cc4c-a4f0-4d2f-bdd0-34dec2571237 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:07.992082 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a4cc4c-a4f0-4d2f-bdd0-34dec2571237 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:08.006924 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 364/729] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:30:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:08.021464 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a4cc4c-a4f0-4d2f-bdd0-34dec2571237 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:08.023434 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb7e6dd-1803-4bda-bc0e-317a141fe083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:08.024413 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7e6dd-1803-4bda-bc0e-317a141fe083 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:08.024870 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7e6dd-1803-4bda-bc0e-317a141fe083 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:08.025414 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7e6dd-1803-4bda-bc0e-317a141fe083 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:08.034363 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb7e6dd-1803-4bda-bc0e-317a141fe083 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:08.036174 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb7e6dd-1803-4bda-bc0e-317a141fe083 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:08.049283 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-dcb7e6dd-1803-4bda-bc0e-317a141fe083 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 14:30:08.049283 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:08.049283 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:08.049283 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:08.049283 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:08.049283 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:08.049283 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:08.049283 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:08.049283 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:08.049283 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:08.049283 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:08.049283 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:08.049283 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:30:08.049283 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:30:08.049283 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:30:08.049283 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:30:08.050665 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:08.050665 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:08.050665 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:30:08.050665 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:30:08.050665 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:30:08.050665 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:30:08.050665 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:08.050665 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:08.050665 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:08.050665 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:08.050665 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:08.050665 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:08.050665 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:08.050665 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:08.050665 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:08.050665 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:08.050665 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:08.050665 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:08.051616 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:08.051616 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:08.051616 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:30:08.051616 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:30:08.051616 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:30:08.051616 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:30:08.051616 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:30:08.051616 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:30:08.051616 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 14:30:08.051616 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:08.054208 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 366/730] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Aug 30 14:30:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:08.061080 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a69f45-9efe-4351-bdc7-336d2f5e9c0d None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:08.061522 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a69f45-9efe-4351-bdc7-336d2f5e9c0d None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:08.061958 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a69f45-9efe-4351-bdc7-336d2f5e9c0d None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:08.063785 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 367/731] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:08 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:08.064746 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a69f45-9efe-4351-bdc7-336d2f5e9c0d None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:08.130644 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a4cc4c-a4f0-4d2f-bdd0-34dec2571237 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:08.165643 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4214e91e-025d-42c1-bbbe-7e88a91cf7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:08.166524 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4214e91e-025d-42c1-bbbe-7e88a91cf7cf None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:08.167558 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4214e91e-025d-42c1-bbbe-7e88a91cf7cf None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:08.168181 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4214e91e-025d-42c1-bbbe-7e88a91cf7cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:08.174699 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a69f45-9efe-4351-bdc7-336d2f5e9c0d None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:08.179758 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4214e91e-025d-42c1-bbbe-7e88a91cf7cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:08.181306 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4214e91e-025d-42c1-bbbe-7e88a91cf7cf None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:08.197144 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-4214e91e-025d-42c1-bbbe-7e88a91cf7cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:08.197144 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:08.197144 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:08.197144 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:08.197144 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:08.197144 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:08.197144 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:08.197144 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:08.197144 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:08.197144 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:08.197144 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:08.197144 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:08.197144 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:30:08.197144 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:30:08.197144 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:30:08.197144 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:30:08.198433 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:08.198433 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:08.198433 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:08.198433 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:08.198433 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:08.198433 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:08.198433 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:08.198433 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:08.198433 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:08.198433 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:08.198433 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:08.198433 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:08.198433 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:08.198433 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:08.198433 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:08.198433 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:08.198433 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:30:08.199360 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:30:08.199360 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:08.199360 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:08.203629 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 365/732] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:08.204946 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18465b9b-0d95-4bd5-9fde-fc210d1238c2 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:08.205350 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18465b9b-0d95-4bd5-9fde-fc210d1238c2 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:08.205640 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18465b9b-0d95-4bd5-9fde-fc210d1238c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:08.212893 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:08.225834 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-18465b9b-0d95-4bd5-9fde-fc210d1238c2 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:08.227327 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-18465b9b-0d95-4bd5-9fde-fc210d1238c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:08.229919 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349f2fcb-0314-43f6-828e-f60a31a417bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:08.231049 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f2fcb-0314-43f6-828e-f60a31a417bb None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:08.231645 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f2fcb-0314-43f6-828e-f60a31a417bb None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:08.232118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f2fcb-0314-43f6-828e-f60a31a417bb None admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:08.233085 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349f2fcb-0314-43f6-828e-f60a31a417bb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:08.233563 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349f2fcb-0314-43f6-828e-f60a31a417bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:08.235229 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349f2fcb-0314-43f6-828e-f60a31a417bb None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:08.260096 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 368/733] 10.222.0.30 () {56 vars in 1149 bytes} [Tue Aug 30 14:30:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:08.261439 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349f2fcb-0314-43f6-828e-f60a31a417bb None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:08.287579 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:08.308037 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 366/734] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:30:08.308037 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18465b9b-0d95-4bd5-9fde-fc210d1238c2 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:08.407800 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18465b9b-0d95-4bd5-9fde-fc210d1238c2 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:08.421728 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349f2fcb-0314-43f6-828e-f60a31a417bb None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:08.444182 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e456622e-7b55-4544-8785-1cac6ffbcb97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:08.444792 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e456622e-7b55-4544-8785-1cac6ffbcb97 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:08.445071 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e456622e-7b55-4544-8785-1cac6ffbcb97 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:08.445340 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e456622e-7b55-4544-8785-1cac6ffbcb97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:08.448238 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:08.457915 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c764588-b4d4-41d5-853c-b1fb4b5cd6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:08.458527 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c764588-b4d4-41d5-853c-b1fb4b5cd6d2 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:08.458802 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c764588-b4d4-41d5-853c-b1fb4b5cd6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:08.459071 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c764588-b4d4-41d5-853c-b1fb4b5cd6d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:08.459749 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c764588-b4d4-41d5-853c-b1fb4b5cd6d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:08.460051 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c764588-b4d4-41d5-853c-b1fb4b5cd6d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:08.461418 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c764588-b4d4-41d5-853c-b1fb4b5cd6d2 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:08.478476 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 369/735] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:30:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:08.480794 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6b210-9b02-4b4b-8b4e-844b97d65f3e None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:08.481031 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6b210-9b02-4b4b-8b4e-844b97d65f3e None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:08.481309 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6b210-9b02-4b4b-8b4e-844b97d65f3e None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:08.482583 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 370/736] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:08.487352 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d6b210-9b02-4b4b-8b4e-844b97d65f3e None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:08.519891 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e456622e-7b55-4544-8785-1cac6ffbcb97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:08.521405 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e456622e-7b55-4544-8785-1cac6ffbcb97 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:08.574944 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:08.597856 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d6b210-9b02-4b4b-8b4e-844b97d65f3e None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:08.627068 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 367/737] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:30:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 14:30:08.628395 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a252b43-4850-4508-902a-83d72e6bbf1b None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:08.628678 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a252b43-4850-4508-902a-83d72e6bbf1b None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:08.628949 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a252b43-4850-4508-902a-83d72e6bbf1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:08.634564 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b3f946-afef-4755-a08a-7e4b7a35cd30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:08.636010 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3f946-afef-4755-a08a-7e4b7a35cd30 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:08.636448 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3f946-afef-4755-a08a-7e4b7a35cd30 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:08.637057 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3f946-afef-4755-a08a-7e4b7a35cd30 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:08.645807 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b3f946-afef-4755-a08a-7e4b7a35cd30 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:08.647492 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b3f946-afef-4755-a08a-7e4b7a35cd30 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:08.648656 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-1a252b43-4850-4508-902a-83d72e6bbf1b None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:08.649834 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-1a252b43-4850-4508-902a-83d72e6bbf1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:08.659529 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-86b3f946-afef-4755-a08a-7e4b7a35cd30 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 14:30:08.659529 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:08.659529 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:08.659529 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:08.659529 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:08.659529 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:08.659529 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:08.659529 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:08.659529 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:08.659529 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:08.659529 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:08.659529 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:08.659529 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:30:08.659529 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:30:08.659529 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:30:08.659529 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:30:08.660520 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:08.660520 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:08.660520 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:30:08.660520 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:30:08.660520 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:30:08.660520 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:30:08.660520 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:08.660520 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:08.660520 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:08.660520 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:08.660520 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:08.660520 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:08.660520 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:08.660520 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:08.660520 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:08.660520 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:08.660520 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:08.660520 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:08.661467 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:08.661467 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:08.661467 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:30:08.661467 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:30:08.661467 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:30:08.661467 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:30:08.661467 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:30:08.661467 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:30:08.661467 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 14:30:08.661467 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:08.662429 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 371/738] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 30 14:30:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:08.662986 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b3f946-afef-4755-a08a-7e4b7a35cd30 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:08.680058 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:08.724999 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 368/739] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:30:08.725972 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a252b43-4850-4508-902a-83d72e6bbf1b None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:08.798721 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b3f946-afef-4755-a08a-7e4b7a35cd30 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:08.839464 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a252b43-4850-4508-902a-83d72e6bbf1b None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:08.846568 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c63c76-5eb4-403d-b7f3-3c2e9878cce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:08.847976 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c63c76-5eb4-403d-b7f3-3c2e9878cce7 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:08.848410 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c63c76-5eb4-403d-b7f3-3c2e9878cce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:08.848834 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c63c76-5eb4-403d-b7f3-3c2e9878cce7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:08.862226 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:08.866839 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c63c76-5eb4-403d-b7f3-3c2e9878cce7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:08.869583 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c63c76-5eb4-403d-b7f3-3c2e9878cce7 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:08.878834 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1c44bb-7604-47ac-b2a7-677e9c7b396c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:08.879527 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1c44bb-7604-47ac-b2a7-677e9c7b396c None admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:08.879805 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1c44bb-7604-47ac-b2a7-677e9c7b396c None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:08.880072 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1c44bb-7604-47ac-b2a7-677e9c7b396c None admin] PATH_INFO: `/v3/services` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:08.880721 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1c44bb-7604-47ac-b2a7-677e9c7b396c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:08.882100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1c44bb-7604-47ac-b2a7-677e9c7b396c None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:08.887600 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-c0c63c76-5eb4-403d-b7f3-3c2e9878cce7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:08.887600 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:08.887600 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:08.887600 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:08.887600 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:08.887600 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:08.887600 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:08.887600 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:08.887600 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:08.887600 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:08.887600 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:08.887600 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:08.887600 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:30:08.887600 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:30:08.887600 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:30:08.887600 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:30:08.888548 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:08.888548 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:08.888548 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:08.888548 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:08.888548 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:08.888548 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:08.888548 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:08.888548 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:08.888548 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:08.888548 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:08.888548 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:08.888548 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:08.888548 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:08.888548 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:08.888548 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:08.888548 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:08.888548 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:30:08.889463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:30:08.889463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:08.889463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:08.889463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 372/740] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:08.893578 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c63c76-5eb4-403d-b7f3-3c2e9878cce7 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:08.903985 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 369/741] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Aug 30 14:30:08 2022] POST /identity/v3/services => generated 240 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:30:08.905357 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a12329-1e1f-4d14-a032-c62f6efabaa6 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:08.905647 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a12329-1e1f-4d14-a032-c62f6efabaa6 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:08.905917 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a12329-1e1f-4d14-a032-c62f6efabaa6 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:08.907313 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 370/742] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:08.907871 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a12329-1e1f-4d14-a032-c62f6efabaa6 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:08.950321 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:09.020740 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c63c76-5eb4-403d-b7f3-3c2e9878cce7 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:09.064235 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a12329-1e1f-4d14-a032-c62f6efabaa6 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:09.069325 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426c90c9-a2e9-4874-af76-e9ab62f4b348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:09.069947 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c90c9-a2e9-4874-af76-e9ab62f4b348 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:09.070228 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c90c9-a2e9-4874-af76-e9ab62f4b348 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:09.070505 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c90c9-a2e9-4874-af76-e9ab62f4b348 None admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:09.071681 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426c90c9-a2e9-4874-af76-e9ab62f4b348 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:09.072044 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426c90c9-a2e9-4874-af76-e9ab62f4b348 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:09.074202 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426c90c9-a2e9-4874-af76-e9ab62f4b348 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:09.095069 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 373/743] 10.222.0.30 () {56 vars in 1135 bytes} [Tue Aug 30 14:30:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:09.097001 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed2d76c-fa78-4f17-bba0-50419ad82057 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:09.097288 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed2d76c-fa78-4f17-bba0-50419ad82057 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:09.097560 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed2d76c-fa78-4f17-bba0-50419ad82057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:09.106135 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e0421-3a9a-4f1a-8332-f020fc7a59ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:09.107432 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e0421-3a9a-4f1a-8332-f020fc7a59ea None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:09.108020 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e0421-3a9a-4f1a-8332-f020fc7a59ea None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:09.108962 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e0421-3a9a-4f1a-8332-f020fc7a59ea None admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:09.109965 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981e0421-3a9a-4f1a-8332-f020fc7a59ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:09.110406 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981e0421-3a9a-4f1a-8332-f020fc7a59ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:09.111888 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981e0421-3a9a-4f1a-8332-f020fc7a59ea None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:09.120952 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-eed2d76c-fa78-4f17-bba0-50419ad82057 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:09.122234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-eed2d76c-fa78-4f17-bba0-50419ad82057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:09.126155 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 371/744] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:30:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:09.126657 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e0421-3a9a-4f1a-8332-f020fc7a59ea None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:09.179187 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:09.214404 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 374/745] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:09 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 14:30:09.215983 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed2d76c-fa78-4f17-bba0-50419ad82057 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:09.216818 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:09.244590 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e0421-3a9a-4f1a-8332-f020fc7a59ea None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:09.281636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91700603-814f-4eb7-837a-80021b9ccc92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:09.282475 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91700603-814f-4eb7-837a-80021b9ccc92 None admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:09.282880 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91700603-814f-4eb7-837a-80021b9ccc92 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:09.283293 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91700603-814f-4eb7-837a-80021b9ccc92 None admin] PATH_INFO: `/v3/services` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:09.284150 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91700603-814f-4eb7-837a-80021b9ccc92 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:09.285743 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91700603-814f-4eb7-837a-80021b9ccc92 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:09.305450 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 372/746] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Aug 30 14:30:09 2022] POST /identity/v3/services => generated 251 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:30:09.306468 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91700603-814f-4eb7-837a-80021b9ccc92 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:09.344878 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed2d76c-fa78-4f17-bba0-50419ad82057 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:09.396858 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b777042b-97dc-4b65-b31d-4cb3157364a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:09.397779 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b777042b-97dc-4b65-b31d-4cb3157364a1 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:09.398234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b777042b-97dc-4b65-b31d-4cb3157364a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:09.398651 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b777042b-97dc-4b65-b31d-4cb3157364a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:09.463149 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:09.467110 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91700603-814f-4eb7-837a-80021b9ccc92 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:09.492937 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b777042b-97dc-4b65-b31d-4cb3157364a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:09.495784 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b777042b-97dc-4b65-b31d-4cb3157364a1 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:09.524483 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5338a42-ffc7-405b-9eab-6b6074617112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:09.525128 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5338a42-ffc7-405b-9eab-6b6074617112 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:09.525404 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5338a42-ffc7-405b-9eab-6b6074617112 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:09.525679 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5338a42-ffc7-405b-9eab-6b6074617112 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:09.533004 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5338a42-ffc7-405b-9eab-6b6074617112 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:09.534766 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5338a42-ffc7-405b-9eab-6b6074617112 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:09.542701 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:09.548944 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-f5338a42-ffc7-405b-9eab-6b6074617112 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:30:09.548944 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:09.548944 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:09.548944 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:09.548944 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:09.548944 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:09.548944 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:09.548944 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:09.548944 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:09.548944 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:09.548944 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:09.548944 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:09.548944 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:30:09.548944 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:30:09.548944 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:30:09.549965 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:30:09.549965 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:09.549965 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:09.549965 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:09.549965 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:09.549965 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:09.549965 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:09.549965 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:09.549965 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:09.549965 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:09.549965 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:09.549965 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:09.549965 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:09.549965 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:09.549965 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:09.549965 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:09.549965 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:09.550880 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:30:09.550880 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:30:09.550880 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:30:09.550880 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:30:09.550880 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:30:09.550880 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:30:09.550880 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:30:09.550880 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:09.550880 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 373/747] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:30:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:09.550880 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5338a42-ffc7-405b-9eab-6b6074617112 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:09.611426 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 375/748] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:30:09 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:30:09.616921 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b777042b-97dc-4b65-b31d-4cb3157364a1 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:09.689414 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5338a42-ffc7-405b-9eab-6b6074617112 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:09.715600 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:09.726118 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc1e71b-1b01-4c80-a6aa-d003c2eeb6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:09.727038 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc1e71b-1b01-4c80-a6aa-d003c2eeb6db None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:09.727414 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc1e71b-1b01-4c80-a6aa-d003c2eeb6db None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:09.727762 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc1e71b-1b01-4c80-a6aa-d003c2eeb6db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:09.738735 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b777042b-97dc-4b65-b31d-4cb3157364a1 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:09.739639 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc1e71b-1b01-4c80-a6aa-d003c2eeb6db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:09.746146 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-8bc1e71b-1b01-4c80-a6aa-d003c2eeb6db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:30:09.746146 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:09.746146 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:09.746146 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:09.746146 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:09.746146 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:09.746146 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:09.746146 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:09.746146 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:09.746146 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:09.746146 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:09.746146 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:09.746146 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:09.746146 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:09.746146 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:09.746146 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:09.747133 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:09.747133 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:09.747133 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:09.747133 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:09.747133 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:09.747133 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:09.747133 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:09.747133 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:09.747133 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:09.747133 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:09.747133 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:09.747133 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:09.747133 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:09.747133 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:09.747133 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:09.747133 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:09.748018 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:09.748018 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:09.748018 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:09.748018 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:09.748018 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:09.748018 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:09.748018 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:09.748018 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:09.748018 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:09.748018 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:09.748018 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:30:09.748018 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:09.751667 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 374/749] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 30 14:30:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:09.752643 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc1e71b-1b01-4c80-a6aa-d003c2eeb6db None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:09.788723 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a711299a-647a-4c68-ad1b-a7d0a54d75c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:09.789643 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a711299a-647a-4c68-ad1b-a7d0a54d75c4 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:09.790102 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a711299a-647a-4c68-ad1b-a7d0a54d75c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:09.790574 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a711299a-647a-4c68-ad1b-a7d0a54d75c4 None admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:09.792345 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a711299a-647a-4c68-ad1b-a7d0a54d75c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:09.793722 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a711299a-647a-4c68-ad1b-a7d0a54d75c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:09.795400 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a711299a-647a-4c68-ad1b-a7d0a54d75c4 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:09.812353 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 376/750] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 30 14:30:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:09.814586 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a711299a-647a-4c68-ad1b-a7d0a54d75c4 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:09.819547 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:09.874050 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc1e71b-1b01-4c80-a6aa-d003c2eeb6db None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:09.923940 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce91a3a0-8d64-4217-89eb-5e1ca4030b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:09.925298 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce91a3a0-8d64-4217-89eb-5e1ca4030b3a None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:09.926058 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce91a3a0-8d64-4217-89eb-5e1ca4030b3a None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:09.926511 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce91a3a0-8d64-4217-89eb-5e1ca4030b3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:09.935760 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a711299a-647a-4c68-ad1b-a7d0a54d75c4 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:09.941686 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce91a3a0-8d64-4217-89eb-5e1ca4030b3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:09.943366 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce91a3a0-8d64-4217-89eb-5e1ca4030b3a None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:09.964051 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-ce91a3a0-8d64-4217-89eb-5e1ca4030b3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:09.964051 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:09.964051 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:09.964051 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:09.964051 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:09.964051 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:09.964051 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:09.964051 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:09.964051 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:09.964051 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:09.964051 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:09.964051 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:09.964051 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:30:09.964051 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:30:09.964051 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:30:09.964051 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:30:09.965248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:09.965248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:09.965248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:09.965248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:09.965248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:09.965248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:09.965248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:09.965248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:09.965248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:09.965248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:09.965248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:09.965248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:09.965248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:09.965248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:09.965248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:09.965248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:09.965248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:30:09.966177 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:30:09.966177 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:09.966177 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:09.968785 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 375/751] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:09.969785 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce91a3a0-8d64-4217-89eb-5e1ca4030b3a None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:09.987757 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc1e4c-2ed6-4807-932e-0c2dc3a1a8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:09.988884 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc1e4c-2ed6-4807-932e-0c2dc3a1a8ca None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:09.989313 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc1e4c-2ed6-4807-932e-0c2dc3a1a8ca None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:09.989730 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc1e4c-2ed6-4807-932e-0c2dc3a1a8ca None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:09.991178 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cc1e4c-2ed6-4807-932e-0c2dc3a1a8ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:09.991673 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cc1e4c-2ed6-4807-932e-0c2dc3a1a8ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:09.993187 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cc1e4c-2ed6-4807-932e-0c2dc3a1a8ca None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:09.997171 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:10.008634 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 377/752] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:30:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:10.009744 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc1e4c-2ed6-4807-932e-0c2dc3a1a8ca None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:10.106099 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce91a3a0-8d64-4217-89eb-5e1ca4030b3a None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:10.118287 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:10.132935 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc1e4c-2ed6-4807-932e-0c2dc3a1a8ca None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:10.148313 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122dd913-cc43-4a5b-a997-9bdf5b255830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:10.149180 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122dd913-cc43-4a5b-a997-9bdf5b255830 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:10.149634 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122dd913-cc43-4a5b-a997-9bdf5b255830 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:10.150081 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122dd913-cc43-4a5b-a997-9bdf5b255830 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:10.151514 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122dd913-cc43-4a5b-a997-9bdf5b255830 None admin] RBAC: Adding query filter params (role.id=1316db16eac54629a283545f1601a236, scope.project.id=01cf71ca5abf4576b89d10c85af6d2de, user.id=f60bee471e954932a928ab2cf7d66c82) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:10.151972 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122dd913-cc43-4a5b-a997-9bdf5b255830 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:10.153640 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122dd913-cc43-4a5b-a997-9bdf5b255830 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:10.159848 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 376/753] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:30:09 2022] GET /identity/v3/role_assignments?user.id=f60bee471e954932a928ab2cf7d66c82&scope.project.id=01cf71ca5abf4576b89d10c85af6d2de&role.id=1316db16eac54629a283545f1601a236 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:10.160875 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122dd913-cc43-4a5b-a997-9bdf5b255830 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:10.174202 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fbfeac-5e10-467a-85ef-4242e56376d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:10.174866 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fbfeac-5e10-467a-85ef-4242e56376d6 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:10.175231 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fbfeac-5e10-467a-85ef-4242e56376d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:10.175542 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fbfeac-5e10-467a-85ef-4242e56376d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:10.176408 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fbfeac-5e10-467a-85ef-4242e56376d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:10.176649 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fbfeac-5e10-467a-85ef-4242e56376d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:10.178082 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fbfeac-5e10-467a-85ef-4242e56376d6 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:10.194604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 378/754] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:30:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:10.200978 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fbfeac-5e10-467a-85ef-4242e56376d6 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:10.275483 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:10.285619 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122dd913-cc43-4a5b-a997-9bdf5b255830 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:10.327270 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fbfeac-5e10-467a-85ef-4242e56376d6 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:10.331676 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3bc3a0-3cda-4c18-9f99-15ad33c8fb5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:10.332474 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3bc3a0-3cda-4c18-9f99-15ad33c8fb5f None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:10.332760 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3bc3a0-3cda-4c18-9f99-15ad33c8fb5f None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:10.333031 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3bc3a0-3cda-4c18-9f99-15ad33c8fb5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:10.345571 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3bc3a0-3cda-4c18-9f99-15ad33c8fb5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:10.347054 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3bc3a0-3cda-4c18-9f99-15ad33c8fb5f None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:10.362679 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-8b3bc3a0-3cda-4c18-9f99-15ad33c8fb5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:10.362679 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:10.362679 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:10.362679 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:10.362679 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:10.362679 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:10.362679 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:10.362679 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:10.362679 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:10.362679 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:10.362679 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:10.362679 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:10.362679 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:30:10.362679 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:30:10.362679 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:30:10.362679 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:30:10.363898 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:10.363898 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:10.363898 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:10.363898 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:10.363898 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:10.363898 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:10.363898 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:10.363898 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:10.363898 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:10.363898 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:10.363898 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:10.363898 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:10.363898 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:10.363898 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:10.363898 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:10.363898 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:10.363898 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:30:10.365003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:30:10.365003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:10.365003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:10.375237 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 377/755] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:10.375444 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cabadd4-f4f1-43eb-ab9c-a87cf5e52099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:10.375574 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cabadd4-f4f1-43eb-ab9c-a87cf5e52099 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:10.375765 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cabadd4-f4f1-43eb-ab9c-a87cf5e52099 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:10.376044 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cabadd4-f4f1-43eb-ab9c-a87cf5e52099 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:10.380471 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3bc3a0-3cda-4c18-9f99-15ad33c8fb5f None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:10.400570 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cabadd4-f4f1-43eb-ab9c-a87cf5e52099 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:10.402367 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cabadd4-f4f1-43eb-ab9c-a87cf5e52099 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:10.425168 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-4cabadd4-f4f1-43eb-ab9c-a87cf5e52099 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:30:10.425168 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:10.425168 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:10.425168 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:10.425168 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:10.425168 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:10.425168 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:10.425168 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:10.425168 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:10.425168 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:10.425168 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:10.425168 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:10.425168 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:30:10.425168 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:30:10.425168 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:30:10.426300 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:30:10.426300 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:10.426300 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:10.426300 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:10.426300 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:10.426300 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:10.426300 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:10.426300 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:10.426300 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:10.426300 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:10.426300 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:10.426300 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:10.426300 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:10.426300 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:10.426300 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:10.426300 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:10.426300 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:10.427497 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:30:10.427497 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:30:10.427497 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:30:10.427497 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:30:10.427497 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:30:10.427497 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:30:10.427497 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:30:10.427497 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:10.436328 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 379/756] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:30:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:30:10.441961 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cabadd4-f4f1-43eb-ab9c-a87cf5e52099 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:10.463749 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:10.497798 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3bc3a0-3cda-4c18-9f99-15ad33c8fb5f None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:10.563247 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95989d17-f5bd-4f15-b1d6-5f1782a48d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:10.563874 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95989d17-f5bd-4f15-b1d6-5f1782a48d92 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:10.564156 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95989d17-f5bd-4f15-b1d6-5f1782a48d92 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:10.564426 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95989d17-f5bd-4f15-b1d6-5f1782a48d92 None admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:10.565458 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95989d17-f5bd-4f15-b1d6-5f1782a48d92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:10.565458 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95989d17-f5bd-4f15-b1d6-5f1782a48d92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:10.567626 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95989d17-f5bd-4f15-b1d6-5f1782a48d92 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:10.598703 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 378/757] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:30:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:30:10.601115 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95989d17-f5bd-4f15-b1d6-5f1782a48d92 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:10.685599 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cabadd4-f4f1-43eb-ab9c-a87cf5e52099 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:10.727556 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:10.750848 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b14f60-3a2a-4d67-abfa-2697c286f380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:10.752870 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b14f60-3a2a-4d67-abfa-2697c286f380 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:10.753457 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b14f60-3a2a-4d67-abfa-2697c286f380 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:10.753874 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b14f60-3a2a-4d67-abfa-2697c286f380 None admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:10.754716 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b14f60-3a2a-4d67-abfa-2697c286f380 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:10.755199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b14f60-3a2a-4d67-abfa-2697c286f380 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:10.763466 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b14f60-3a2a-4d67-abfa-2697c286f380 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:10.784382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:10.801633 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 380/758] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 30 14:30:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:30:10.812293 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b14f60-3a2a-4d67-abfa-2697c286f380 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:10.823076 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95989d17-f5bd-4f15-b1d6-5f1782a48d92 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:10.881916 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa81acf1-a565-4bef-b3c8-f5793765f0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:10.882914 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa81acf1-a565-4bef-b3c8-f5793765f0f8 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:10.883416 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa81acf1-a565-4bef-b3c8-f5793765f0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:10.884928 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa81acf1-a565-4bef-b3c8-f5793765f0f8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:10.897035 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa81acf1-a565-4bef-b3c8-f5793765f0f8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:10.904363 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa81acf1-a565-4bef-b3c8-f5793765f0f8 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:10.933816 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-aa81acf1-a565-4bef-b3c8-f5793765f0f8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:30:10.933816 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:10.933816 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:10.933816 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:10.933816 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:10.933816 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:10.933816 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:10.933816 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:10.933816 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:10.933816 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:10.933816 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:10.933816 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:10.933816 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:30:10.933816 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:30:10.933816 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:30:10.933816 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:30:10.935055 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:10.935055 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:10.935055 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:30:10.935055 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:30:10.935055 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:30:10.935055 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:30:10.935055 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:10.935055 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:10.935055 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:10.935055 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:10.935055 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:10.935055 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:10.935055 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:10.935055 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:10.935055 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:10.935055 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:10.935055 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:10.935055 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:10.936838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:10.936838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:10.936838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:30:10.936838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:30:10.936838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:30:10.936838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:30:10.936838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:30:10.936838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:30:10.936838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:30:10.936838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:10.940154 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 379/759] 10.222.0.30 () {56 vars in 1093 bytes} [Tue Aug 30 14:30:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:10.949703 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa81acf1-a565-4bef-b3c8-f5793765f0f8 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:11.043997 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b14f60-3a2a-4d67-abfa-2697c286f380 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:11.124908 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4415d6d-9852-420b-b981-1b99b620f5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:11.125911 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4415d6d-9852-420b-b981-1b99b620f5ba None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:11.126401 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4415d6d-9852-420b-b981-1b99b620f5ba None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:11.126877 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4415d6d-9852-420b-b981-1b99b620f5ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:11.127819 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4415d6d-9852-420b-b981-1b99b620f5ba None admin] RBAC: Adding query filter params (role.id=3cda0615d87d4af0860a2a5e9b4e2c05, scope.project.id=01cf71ca5abf4576b89d10c85af6d2de, user.id=a18b13e3708144e1b5e8df80ec3a4e83) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:11.128396 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4415d6d-9852-420b-b981-1b99b620f5ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:11.130034 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4415d6d-9852-420b-b981-1b99b620f5ba None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:11.136961 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 381/760] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:30:10 2022] GET /identity/v3/role_assignments?user.id=a18b13e3708144e1b5e8df80ec3a4e83&scope.project.id=01cf71ca5abf4576b89d10c85af6d2de&role.id=3cda0615d87d4af0860a2a5e9b4e2c05 => generated 259 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:11.222954 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200bb01b-1925-4918-a507-6df97c541468 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:11.222954 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200bb01b-1925-4918-a507-6df97c541468 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:11.223365 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200bb01b-1925-4918-a507-6df97c541468 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:11.224809 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 382/761] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:11.230364 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f952fc2e-0259-4e2a-b47e-8b460d1e5dfb None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:11.230630 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f952fc2e-0259-4e2a-b47e-8b460d1e5dfb None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:11.230903 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f952fc2e-0259-4e2a-b47e-8b460d1e5dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:11.262689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa81acf1-a565-4bef-b3c8-f5793765f0f8 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:11.277356 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-f952fc2e-0259-4e2a-b47e-8b460d1e5dfb None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:11.278957 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-f952fc2e-0259-4e2a-b47e-8b460d1e5dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:11.356004 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c587b2-76cd-4e35-9555-2134af484c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:11.356004 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c587b2-76cd-4e35-9555-2134af484c20 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:11.356004 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c587b2-76cd-4e35-9555-2134af484c20 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:11.356004 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c587b2-76cd-4e35-9555-2134af484c20 None admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:11.356473 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c587b2-76cd-4e35-9555-2134af484c20 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:11.356473 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c587b2-76cd-4e35-9555-2134af484c20 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:11.356790 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c587b2-76cd-4e35-9555-2134af484c20 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:11.363046 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:11.397671 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 380/762] 10.222.0.30 () {56 vars in 1133 bytes} [Tue Aug 30 14:30:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 447 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:11.401363 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c587b2-76cd-4e35-9555-2134af484c20 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:11.410008 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:11.425128 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 383/763] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:30:11.557984 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c587b2-76cd-4e35-9555-2134af484c20 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:11.580349 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734e3fc-37db-4101-a3d6-4ed21ffb95cd None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:11.580687 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734e3fc-37db-4101-a3d6-4ed21ffb95cd None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:11.581062 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734e3fc-37db-4101-a3d6-4ed21ffb95cd None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:11.584460 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 384/764] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:11.591234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc53ee4-9d5b-4444-acb0-ab9c117cc6d9 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:11.591234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc53ee4-9d5b-4444-acb0-ab9c117cc6d9 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:11.591234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc53ee4-9d5b-4444-acb0-ab9c117cc6d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:11.624702 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-edc53ee4-9d5b-4444-acb0-ab9c117cc6d9 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:11.625884 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-edc53ee4-9d5b-4444-acb0-ab9c117cc6d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:11.631229 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce2bae1-6c23-4831-9028-198039f25b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:11.631385 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce2bae1-6c23-4831-9028-198039f25b8a None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:11.631630 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce2bae1-6c23-4831-9028-198039f25b8a None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:11.632623 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce2bae1-6c23-4831-9028-198039f25b8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:11.655631 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce2bae1-6c23-4831-9028-198039f25b8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:11.657204 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce2bae1-6c23-4831-9028-198039f25b8a None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:11.685525 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-2ce2bae1-6c23-4831-9028-198039f25b8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:11.685525 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:11.685525 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:11.685525 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:11.685525 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:11.685525 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:11.685525 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:11.685525 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:11.685525 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:11.685525 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:11.685525 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:11.685525 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:11.685525 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:30:11.685525 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:30:11.685525 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:30:11.685525 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:30:11.686549 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:11.686549 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:11.686549 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:11.686549 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:11.686549 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:11.686549 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:11.686549 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:11.686549 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:11.686549 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:11.686549 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:11.686549 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:11.686549 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:11.686549 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:11.686549 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:11.686549 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:11.686549 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:11.686549 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:30:11.687520 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:30:11.687520 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:11.687520 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:11.688423 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 381/765] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:11.704602 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce2bae1-6c23-4831-9028-198039f25b8a None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:11.749892 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 385/766] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:30:11.778245 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc53ee4-9d5b-4444-acb0-ab9c117cc6d9 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:11.830807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:11.887712 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce2bae1-6c23-4831-9028-198039f25b8a None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:11.934590 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad949fc2-ce7c-4548-8c08-936846bff343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:11.935690 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad949fc2-ce7c-4548-8c08-936846bff343 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:11.937606 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad949fc2-ce7c-4548-8c08-936846bff343 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:11.938862 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad949fc2-ce7c-4548-8c08-936846bff343 None admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:11.940492 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad949fc2-ce7c-4548-8c08-936846bff343 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:11.941093 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad949fc2-ce7c-4548-8c08-936846bff343 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:11.943808 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad949fc2-ce7c-4548-8c08-936846bff343 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:11.958450 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 382/767] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:30:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:11.965615 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad949fc2-ce7c-4548-8c08-936846bff343 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:12.041035 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc53ee4-9d5b-4444-acb0-ab9c117cc6d9 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:12.042971 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:12.078342 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad949fc2-ce7c-4548-8c08-936846bff343 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:12.100719 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:12.120648 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b749e8cd-02d0-4610-943c-6d069f82b671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:12.121592 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b749e8cd-02d0-4610-943c-6d069f82b671 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:12.122090 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b749e8cd-02d0-4610-943c-6d069f82b671 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:12.122554 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b749e8cd-02d0-4610-943c-6d069f82b671 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:12.129946 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b749e8cd-02d0-4610-943c-6d069f82b671 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:12.131656 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b749e8cd-02d0-4610-943c-6d069f82b671 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:12.143942 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ed823f-1c0f-4877-9fd8-e7916d19c87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:12.144583 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed823f-1c0f-4877-9fd8-e7916d19c87f None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:12.144867 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed823f-1c0f-4877-9fd8-e7916d19c87f None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:12.145141 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed823f-1c0f-4877-9fd8-e7916d19c87f None admin] PATH_INFO: `/v3/services/network` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:12.145314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-b749e8cd-02d0-4610-943c-6d069f82b671 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:30:12.145314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:12.145314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:12.145314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:12.145314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:12.145314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:12.145314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:12.145314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:12.145314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:12.145314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:12.145314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:12.145314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:12.145314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:30:12.145314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:30:12.145314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:30:12.146232 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:30:12.146232 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:12.146232 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:12.146232 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:12.146232 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:12.146232 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:12.146232 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:12.146232 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:12.146232 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:12.146232 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:12.146232 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:12.146232 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:12.146232 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:12.146232 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:12.146232 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:12.146232 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:12.146232 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:12.147153 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:30:12.147153 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:30:12.147153 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:30:12.147153 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:30:12.147153 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:30:12.147153 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:30:12.147153 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:30:12.147153 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:12.147601 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ed823f-1c0f-4877-9fd8-e7916d19c87f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:12.147601 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ed823f-1c0f-4877-9fd8-e7916d19c87f None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:12.149888 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 383/768] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:30:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:12.155864 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b749e8cd-02d0-4610-943c-6d069f82b671 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:12.156708 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-89ed823f-1c0f-4877-9fd8-e7916d19c87f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 14:30:12.156708 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:12.156708 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:30:12.156708 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:30:12.156708 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:30:12.156708 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:30:12.156708 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:30:12.156708 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:30:12.156708 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 14:30:12.156708 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:12.156708 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:30:12.156708 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:12.156708 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:12.156708 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:12.156708 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:12.156708 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:12.156708 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:12.157709 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:12.157709 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:12.157709 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:12.157709 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:12.157709 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:12.157709 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:12.157709 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:30:12.157709 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:30:12.157709 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:30:12.157709 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:30:12.157709 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:12.157709 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:12.157709 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:12.157709 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:12.157709 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:12.157709 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:12.158563 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:12.158563 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:12.158563 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:12.158563 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:12.158563 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:12.158563 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:12.158563 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:12.158563 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:12.158563 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:12.158563 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:12.158563 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:30:12.158563 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:30:12.158563 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 14:30:12.158563 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:12.162541 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 386/769] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:30:11 2022] GET /identity/v3/services/network => generated 88 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:12.164162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ed823f-1c0f-4877-9fd8-e7916d19c87f None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:12.275379 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b749e8cd-02d0-4610-943c-6d069f82b671 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:12.280549 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ed823f-1c0f-4877-9fd8-e7916d19c87f None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:12.317213 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5bf072-b9d1-45b1-9251-9f6da54eadf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:12.318819 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5bf072-b9d1-45b1-9251-9f6da54eadf1 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:12.319640 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5bf072-b9d1-45b1-9251-9f6da54eadf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:12.320115 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5bf072-b9d1-45b1-9251-9f6da54eadf1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:12.320899 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5bf072-b9d1-45b1-9251-9f6da54eadf1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:12.322550 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5bf072-b9d1-45b1-9251-9f6da54eadf1 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:12.323396 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c80ca96-2fb8-4404-abe0-13bd46cffbb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:12.324099 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c80ca96-2fb8-4404-abe0-13bd46cffbb2 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:12.324381 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c80ca96-2fb8-4404-abe0-13bd46cffbb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:12.324653 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c80ca96-2fb8-4404-abe0-13bd46cffbb2 None admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:12.325411 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c80ca96-2fb8-4404-abe0-13bd46cffbb2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:12.325770 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c80ca96-2fb8-4404-abe0-13bd46cffbb2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:12.327281 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c80ca96-2fb8-4404-abe0-13bd46cffbb2 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:12.334214 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-df5bf072-b9d1-45b1-9251-9f6da54eadf1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 14:30:12.334214 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:12.334214 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:30:12.334214 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:30:12.334214 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:30:12.334214 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:30:12.334214 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:30:12.334214 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:30:12.334214 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 14:30:12.334214 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:12.334214 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:30:12.334214 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:12.334214 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:12.334214 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:12.334214 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:12.334214 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:12.334214 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:12.335236 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:12.335236 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:12.335236 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:12.335236 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:12.335236 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:12.335236 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:12.335236 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:30:12.335236 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:30:12.335236 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:30:12.335236 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:30:12.335236 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:12.335236 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:12.335236 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:12.335236 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:12.335236 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:12.335236 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:12.336083 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:12.336083 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:12.336083 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:12.336083 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:12.336083 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:12.336083 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:12.336083 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:12.336083 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:12.336083 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:12.336083 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:12.336083 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:30:12.336083 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:30:12.336083 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 14:30:12.336083 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:12.336805 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 384/770] 10.222.0.30 () {56 vars in 1129 bytes} [Tue Aug 30 14:30:12 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:12.337957 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5bf072-b9d1-45b1-9251-9f6da54eadf1 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:12.351408 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 387/771] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 30 14:30:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:12.352654 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c80ca96-2fb8-4404-abe0-13bd46cffbb2 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:12.391666 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:12.392790 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:12.465637 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5bf072-b9d1-45b1-9251-9f6da54eadf1 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:12.479544 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c80ca96-2fb8-4404-abe0-13bd46cffbb2 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:12.503180 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66fdbce-0516-4a63-810e-3e2d14f0da79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:12.504123 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66fdbce-0516-4a63-810e-3e2d14f0da79 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:12.504553 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66fdbce-0516-4a63-810e-3e2d14f0da79 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:12.504985 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66fdbce-0516-4a63-810e-3e2d14f0da79 None admin] PATH_INFO: `/v3/services` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:12.506093 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66fdbce-0516-4a63-810e-3e2d14f0da79 None admin] RBAC: Adding query filter params (name=network) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:12.506402 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66fdbce-0516-4a63-810e-3e2d14f0da79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:12.507808 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66fdbce-0516-4a63-810e-3e2d14f0da79 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:12.517606 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 385/772] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Aug 30 14:30:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:12.517606 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66fdbce-0516-4a63-810e-3e2d14f0da79 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:12.527794 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a84610-b59b-4b1f-b1a7-66607efece03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:12.528995 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a84610-b59b-4b1f-b1a7-66607efece03 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:12.529278 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a84610-b59b-4b1f-b1a7-66607efece03 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:12.529553 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a84610-b59b-4b1f-b1a7-66607efece03 None admin] PATH_INFO: `/v3/services` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:12.530510 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a84610-b59b-4b1f-b1a7-66607efece03 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:12.530815 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a84610-b59b-4b1f-b1a7-66607efece03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:12.532236 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a84610-b59b-4b1f-b1a7-66607efece03 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:12.536978 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 388/773] 10.222.0.30 () {56 vars in 1124 bytes} [Tue Aug 30 14:30:12 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:12.538312 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d26176-7f1e-47f5-aea4-ee81221ec146 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:12.538692 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d26176-7f1e-47f5-aea4-ee81221ec146 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:12.539081 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d26176-7f1e-47f5-aea4-ee81221ec146 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:12.540545 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 389/774] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:12.541567 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d26176-7f1e-47f5-aea4-ee81221ec146 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:12.631550 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66fdbce-0516-4a63-810e-3e2d14f0da79 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:12.634777 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:12.645182 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:12.657047 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d26176-7f1e-47f5-aea4-ee81221ec146 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:12.669807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79eab7f-fff8-4d91-8a14-7baf83ed8b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:12.670467 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79eab7f-fff8-4d91-8a14-7baf83ed8b75 None admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:12.670750 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79eab7f-fff8-4d91-8a14-7baf83ed8b75 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:12.671029 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79eab7f-fff8-4d91-8a14-7baf83ed8b75 None admin] PATH_INFO: `/v3/projects/01cf71ca5abf4576b89d10c85af6d2de/users/8878de66a03044e38e507028853a0cd1/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:12.673071 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79eab7f-fff8-4d91-8a14-7baf83ed8b75 None admin] RBAC: Authorizing `identity:create_grant(project_id=01cf71ca5abf4576b89d10c85af6d2de, user_id=8878de66a03044e38e507028853a0cd1, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:12.674476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79eab7f-fff8-4d91-8a14-7baf83ed8b75 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:12.689538 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 386/775] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 30 14:30:12 2022] PUT /identity/v3/projects/01cf71ca5abf4576b89d10c85af6d2de/users/8878de66a03044e38e507028853a0cd1/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:30:12.690266 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79eab7f-fff8-4d91-8a14-7baf83ed8b75 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:12.697878 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc041cd-ee5e-4e78-8ff3-94c4aa492588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:12.698948 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc041cd-ee5e-4e78-8ff3-94c4aa492588 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:12.699483 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc041cd-ee5e-4e78-8ff3-94c4aa492588 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:12.699975 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc041cd-ee5e-4e78-8ff3-94c4aa492588 None admin] PATH_INFO: `/v3/services` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:12.701659 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc041cd-ee5e-4e78-8ff3-94c4aa492588 None admin] RBAC: Adding query filter params (type=network) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:12.702212 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc041cd-ee5e-4e78-8ff3-94c4aa492588 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:12.704604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc041cd-ee5e-4e78-8ff3-94c4aa492588 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:12.712815 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 390/776] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Aug 30 14:30:12 2022] GET /identity/v3/services?type=network => generated 350 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:12.714625 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656e8cde-44f1-4c5a-a1b0-323e83edb335 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:12.715122 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656e8cde-44f1-4c5a-a1b0-323e83edb335 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:12.715661 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656e8cde-44f1-4c5a-a1b0-323e83edb335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:12.745385 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-656e8cde-44f1-4c5a-a1b0-323e83edb335 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:12.747132 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-656e8cde-44f1-4c5a-a1b0-323e83edb335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:12.835060 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 391/777] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:30:12.835726 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656e8cde-44f1-4c5a-a1b0-323e83edb335 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:12.842673 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79eab7f-fff8-4d91-8a14-7baf83ed8b75 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:12.893610 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d62fca4-81ec-4302-b543-c1cfb1ca4bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:12.895010 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62fca4-81ec-4302-b543-c1cfb1ca4bb5 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:12.896810 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62fca4-81ec-4302-b543-c1cfb1ca4bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:12.897339 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62fca4-81ec-4302-b543-c1cfb1ca4bb5 None admin] PATH_INFO: `/v3/services` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:12.898588 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d62fca4-81ec-4302-b543-c1cfb1ca4bb5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:12.899404 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d62fca4-81ec-4302-b543-c1cfb1ca4bb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:12.901302 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d62fca4-81ec-4302-b543-c1cfb1ca4bb5 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:12.908613 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 387/778] 10.222.0.30 () {56 vars in 1124 bytes} [Tue Aug 30 14:30:12 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:12.916680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d62fca4-81ec-4302-b543-c1cfb1ca4bb5 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:12.959561 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:12.973818 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:12.992514 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656e8cde-44f1-4c5a-a1b0-323e83edb335 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:13.051052 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f374e458-8784-4099-acdc-8c2764b09ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:13.052765 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f374e458-8784-4099-acdc-8c2764b09ba1 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:13.053457 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f374e458-8784-4099-acdc-8c2764b09ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:13.053925 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f374e458-8784-4099-acdc-8c2764b09ba1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:13.054816 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f374e458-8784-4099-acdc-8c2764b09ba1 None admin] RBAC: Adding query filter params (interface=public, service_id=05f9480721914cd3b5f46559c15cb69d, region_id=RegionOne) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:13.055339 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f374e458-8784-4099-acdc-8c2764b09ba1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:13.056977 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f374e458-8784-4099-acdc-8c2764b09ba1 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:13.069404 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 392/779] 10.222.0.30 () {56 vars in 1251 bytes} [Tue Aug 30 14:30:12 2022] GET /identity/v3/endpoints?service_id=05f9480721914cd3b5f46559c15cb69d&interface=public®ion_id=RegionOne => generated 194 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:13.085279 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f374e458-8784-4099-acdc-8c2764b09ba1 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:13.139295 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d62fca4-81ec-4302-b543-c1cfb1ca4bb5 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:13.179808 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d0c98-48f3-437a-939b-9b35b8365d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:13.180457 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d0c98-48f3-437a-939b-9b35b8365d38 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:13.180745 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d0c98-48f3-437a-939b-9b35b8365d38 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:13.181022 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d0c98-48f3-437a-939b-9b35b8365d38 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:13.181929 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000d0c98-48f3-437a-939b-9b35b8365d38 None admin] RBAC: Adding query filter params (interface=public, service_id=8ac8c8736fdd431e8e138a6261ed6b3f, region_id=RegionOne) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:13.182390 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000d0c98-48f3-437a-939b-9b35b8365d38 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:13.183622 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000d0c98-48f3-437a-939b-9b35b8365d38 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:13.195264 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 388/780] 10.222.0.30 () {56 vars in 1251 bytes} [Tue Aug 30 14:30:12 2022] GET /identity/v3/endpoints?service_id=8ac8c8736fdd431e8e138a6261ed6b3f&interface=public®ion_id=RegionOne => generated 194 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:13.196660 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d0c98-48f3-437a-939b-9b35b8365d38 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:13.256579 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f374e458-8784-4099-acdc-8c2764b09ba1 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:13.304905 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195f545d-5bee-485c-a4cf-d37e86695f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:13.306326 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195f545d-5bee-485c-a4cf-d37e86695f1e None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:13.306783 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195f545d-5bee-485c-a4cf-d37e86695f1e None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:13.307830 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195f545d-5bee-485c-a4cf-d37e86695f1e None admin] PATH_INFO: `/v3/services` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:13.309169 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195f545d-5bee-485c-a4cf-d37e86695f1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:13.310983 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195f545d-5bee-485c-a4cf-d37e86695f1e None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:13.325341 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 393/781] 10.222.0.30 () {56 vars in 1087 bytes} [Tue Aug 30 14:30:13 2022] GET /identity/v3/services => generated 772 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:13.325341 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195f545d-5bee-485c-a4cf-d37e86695f1e None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:13.331605 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:13.345483 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d0c98-48f3-437a-939b-9b35b8365d38 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:13.357382 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:13.397113 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503082dc-626e-412f-a9f4-f2676fd46654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:13.398203 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503082dc-626e-412f-a9f4-f2676fd46654 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:13.398641 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503082dc-626e-412f-a9f4-f2676fd46654 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:13.399278 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503082dc-626e-412f-a9f4-f2676fd46654 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:13.415545 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503082dc-626e-412f-a9f4-f2676fd46654 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:13.423336 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-503082dc-626e-412f-a9f4-f2676fd46654 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:30:13.423336 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:13.423336 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:13.423336 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:13.423336 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:13.423336 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:13.423336 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:13.423336 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:13.423336 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:13.423336 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:13.423336 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:13.423336 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:13.423336 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:13.423336 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:13.423336 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:13.423336 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:13.424535 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:13.424535 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:13.424535 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:13.424535 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:13.424535 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:13.424535 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:13.424535 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:13.424535 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:13.424535 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:13.424535 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:13.424535 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:13.424535 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:13.424535 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:13.424535 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:13.424535 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:13.424535 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:13.425597 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:13.425597 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:13.425597 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:13.425597 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:13.425597 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:13.425597 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:13.425597 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:13.425597 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:13.425597 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:13.425597 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:13.425597 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:30:13.425597 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:13.433615 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 389/782] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 30 14:30:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:13.433615 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed793a-4ff3-40ed-8cc0-035156dc7fc2 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:13.433615 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed793a-4ff3-40ed-8cc0-035156dc7fc2 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:13.434192 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed793a-4ff3-40ed-8cc0-035156dc7fc2 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:13.435738 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 390/783] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:13.436512 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed793a-4ff3-40ed-8cc0-035156dc7fc2 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:13.487442 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195f545d-5bee-485c-a4cf-d37e86695f1e None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:13.540553 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc65801-c255-4d4c-969e-1f8d46c44b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:13.542759 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc65801-c255-4d4c-969e-1f8d46c44b41 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:13.543254 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc65801-c255-4d4c-969e-1f8d46c44b41 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:13.543713 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc65801-c255-4d4c-969e-1f8d46c44b41 None admin] PATH_INFO: `/v3/services` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:13.544903 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc65801-c255-4d4c-969e-1f8d46c44b41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:13.547094 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc65801-c255-4d4c-969e-1f8d46c44b41 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:13.554011 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 394/784] 10.222.0.30 () {56 vars in 1087 bytes} [Tue Aug 30 14:30:13 2022] GET /identity/v3/services => generated 772 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:13.560800 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9783811-98f6-40ce-9a24-4739ab120d7e None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:13.561703 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9783811-98f6-40ce-9a24-4739ab120d7e None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:13.561703 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9783811-98f6-40ce-9a24-4739ab120d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:13.591949 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed793a-4ff3-40ed-8cc0-035156dc7fc2 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:13.606434 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-f9783811-98f6-40ce-9a24-4739ab120d7e None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:13.608311 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-f9783811-98f6-40ce-9a24-4739ab120d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:13.687320 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d59c327-97f1-4415-931f-8fbeeef64681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:13.687320 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d59c327-97f1-4415-931f-8fbeeef64681 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:13.687320 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d59c327-97f1-4415-931f-8fbeeef64681 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:13.687320 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d59c327-97f1-4415-931f-8fbeeef64681 None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:13.688034 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d59c327-97f1-4415-931f-8fbeeef64681 None admin] RBAC: Adding query filter params (name=service) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:13.688034 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d59c327-97f1-4415-931f-8fbeeef64681 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:13.691236 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d59c327-97f1-4415-931f-8fbeeef64681 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:13.715391 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 391/785] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 30 14:30:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:13.717376 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d59c327-97f1-4415-931f-8fbeeef64681 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:13.737860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:13.753886 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 395/786] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:30:13.840351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d59c327-97f1-4415-931f-8fbeeef64681 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:13.878037 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeaa80a-6f12-4757-b7b2-659504496188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:13.878829 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeaa80a-6f12-4757-b7b2-659504496188 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:13.879159 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeaa80a-6f12-4757-b7b2-659504496188 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:13.879461 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeaa80a-6f12-4757-b7b2-659504496188 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:13.890924 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeaa80a-6f12-4757-b7b2-659504496188 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:13.892440 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeaa80a-6f12-4757-b7b2-659504496188 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:13.910239 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-9aeaa80a-6f12-4757-b7b2-659504496188 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:13.910239 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:13.910239 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:13.910239 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:13.910239 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:13.910239 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:13.910239 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:13.910239 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:13.910239 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:13.910239 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:13.910239 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:13.910239 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:13.910239 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:30:13.910239 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:30:13.910239 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:30:13.910239 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:30:13.911286 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:13.911286 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:13.911286 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:13.911286 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:13.911286 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:13.911286 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:13.911286 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:13.911286 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:13.911286 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:13.911286 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:13.911286 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:13.911286 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:13.911286 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:13.911286 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:13.911286 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:13.911286 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:13.911286 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:30:13.912188 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:30:13.912188 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:13.912188 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:13.912188 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 392/787] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:13.919722 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9783811-98f6-40ce-9a24-4739ab120d7e None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:13.939279 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:14.116179 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeaa80a-6f12-4757-b7b2-659504496188 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:14.188106 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:14.210137 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9783811-98f6-40ce-9a24-4739ab120d7e None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:14.257207 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639392d0-c3c6-4b69-a180-dee14ea67c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:14.257874 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639392d0-c3c6-4b69-a180-dee14ea67c83 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:14.258099 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639392d0-c3c6-4b69-a180-dee14ea67c83 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:14.258352 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639392d0-c3c6-4b69-a180-dee14ea67c83 None admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:14.259262 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639392d0-c3c6-4b69-a180-dee14ea67c83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:14.259575 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639392d0-c3c6-4b69-a180-dee14ea67c83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:14.261810 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639392d0-c3c6-4b69-a180-dee14ea67c83 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:14.275232 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 396/788] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:30:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:14.283366 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeaa80a-6f12-4757-b7b2-659504496188 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:14.287586 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639392d0-c3c6-4b69-a180-dee14ea67c83 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:14.367189 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:14.367814 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9d8821-db84-46a0-b0a6-d21bc8048e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:14.368707 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d8821-db84-46a0-b0a6-d21bc8048e8c None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:14.369191 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d8821-db84-46a0-b0a6-d21bc8048e8c None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:14.369756 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d8821-db84-46a0-b0a6-d21bc8048e8c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:14.396311 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9d8821-db84-46a0-b0a6-d21bc8048e8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:14.413001 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-3f9d8821-db84-46a0-b0a6-d21bc8048e8c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:30:14.413001 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:14.413001 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:14.413001 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:14.413001 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:14.413001 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:14.413001 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:14.413001 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:14.413001 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:14.413001 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:14.413001 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:14.413001 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:14.413001 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:14.413001 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:14.413001 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:14.413001 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:14.413991 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:14.413991 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:14.413991 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:14.413991 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:14.413991 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:14.413991 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:14.413991 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:14.413991 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:14.413991 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:14.413991 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:14.413991 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:14.413991 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:14.413991 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:14.413991 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:14.413991 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:14.413991 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:14.414874 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:14.414874 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:14.414874 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:14.414874 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:14.414874 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:14.414874 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:14.414874 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:14.414874 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:14.414874 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:14.414874 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:14.414874 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:30:14.414874 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:14.414874 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 393/789] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 30 14:30:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:30:14.415628 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639392d0-c3c6-4b69-a180-dee14ea67c83 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:14.420213 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9d8821-db84-46a0-b0a6-d21bc8048e8c None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:14.447857 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd02aef-cdd2-4f15-84aa-6ce985a68a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:14.448557 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd02aef-cdd2-4f15-84aa-6ce985a68a22 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:14.448837 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd02aef-cdd2-4f15-84aa-6ce985a68a22 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:14.449110 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd02aef-cdd2-4f15-84aa-6ce985a68a22 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:14.457740 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd02aef-cdd2-4f15-84aa-6ce985a68a22 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:14.459173 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd02aef-cdd2-4f15-84aa-6ce985a68a22 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:14.473000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-5dd02aef-cdd2-4f15-84aa-6ce985a68a22 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 14:30:14.473000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:14.473000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:14.473000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:14.473000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:14.473000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:14.473000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:14.473000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:14.473000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:14.473000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:14.473000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:14.473000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:14.473000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:30:14.473000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:30:14.473000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:30:14.473000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:30:14.475157 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:14.475157 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:14.475157 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:30:14.475157 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:30:14.475157 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:30:14.475157 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:30:14.475157 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:14.475157 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:14.475157 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:14.475157 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:14.475157 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:14.475157 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:14.475157 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:14.475157 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:14.475157 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:14.475157 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:14.475157 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:14.475157 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:14.477554 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:14.477554 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:14.477554 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:30:14.477554 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:30:14.477554 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:30:14.477554 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:30:14.477554 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:30:14.477554 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:30:14.477554 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 14:30:14.477554 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:14.477554 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 397/790] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Aug 30 14:30:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:14.485938 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd02aef-cdd2-4f15-84aa-6ce985a68a22 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:14.597073 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd02aef-cdd2-4f15-84aa-6ce985a68a22 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:14.626241 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:14.640838 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e370455d-da6b-4fe7-b571-52ed59470441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:14.641450 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370455d-da6b-4fe7-b571-52ed59470441 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:14.641728 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370455d-da6b-4fe7-b571-52ed59470441 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:14.642606 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370455d-da6b-4fe7-b571-52ed59470441 None admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:14.643372 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e370455d-da6b-4fe7-b571-52ed59470441 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:14.643678 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e370455d-da6b-4fe7-b571-52ed59470441 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:14.645112 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e370455d-da6b-4fe7-b571-52ed59470441 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:14.682639 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 398/791] 10.222.0.30 () {56 vars in 1149 bytes} [Tue Aug 30 14:30:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:14.692187 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e370455d-da6b-4fe7-b571-52ed59470441 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:14.707202 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9d8821-db84-46a0-b0a6-d21bc8048e8c None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:14.735676 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:14.747822 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86848509-7809-41e4-a893-ae48dfabb1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:14.748758 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86848509-7809-41e4-a893-ae48dfabb1b8 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:14.749245 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86848509-7809-41e4-a893-ae48dfabb1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:14.749742 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86848509-7809-41e4-a893-ae48dfabb1b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:14.750665 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86848509-7809-41e4-a893-ae48dfabb1b8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:14.751170 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86848509-7809-41e4-a893-ae48dfabb1b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:14.752774 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86848509-7809-41e4-a893-ae48dfabb1b8 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:14.775173 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 394/792] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:30:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:14.784407 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86848509-7809-41e4-a893-ae48dfabb1b8 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:14.866511 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e370455d-da6b-4fe7-b571-52ed59470441 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:14.909089 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86848509-7809-41e4-a893-ae48dfabb1b8 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:14.922473 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acca842-397b-4dd0-bc35-b17a0bfca477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:14.923700 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acca842-397b-4dd0-bc35-b17a0bfca477 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:14.924218 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acca842-397b-4dd0-bc35-b17a0bfca477 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:14.924915 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acca842-397b-4dd0-bc35-b17a0bfca477 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:14.940832 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acca842-397b-4dd0-bc35-b17a0bfca477 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:14.943145 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acca842-397b-4dd0-bc35-b17a0bfca477 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:14.947468 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828defc5-f243-43a5-857a-a3ec41f0bb89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:14.948761 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828defc5-f243-43a5-857a-a3ec41f0bb89 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:14.949116 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828defc5-f243-43a5-857a-a3ec41f0bb89 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:14.949504 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828defc5-f243-43a5-857a-a3ec41f0bb89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:14.960218 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828defc5-f243-43a5-857a-a3ec41f0bb89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:14.961762 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828defc5-f243-43a5-857a-a3ec41f0bb89 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:14.962746 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-1acca842-397b-4dd0-bc35-b17a0bfca477 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:14.962746 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:14.962746 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:14.962746 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:14.962746 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:14.962746 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:14.962746 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:14.962746 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:14.962746 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:14.962746 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:14.962746 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:14.962746 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:14.962746 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:30:14.962746 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:30:14.962746 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:30:14.962746 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:30:14.964225 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:14.964225 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:14.964225 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:14.964225 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:14.964225 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:14.964225 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:14.964225 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:14.964225 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:14.964225 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:14.964225 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:14.964225 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:14.964225 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:14.964225 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:14.964225 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:14.964225 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:14.964225 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:14.964225 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:30:14.965148 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:30:14.965148 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:14.965148 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:14.967051 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 399/793] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:14.968399 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55bce23-3a9a-4c6d-a9a1-ef51522a1190 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:14.968852 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55bce23-3a9a-4c6d-a9a1-ef51522a1190 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:14.969277 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55bce23-3a9a-4c6d-a9a1-ef51522a1190 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:14.974903 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 400/794] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:14.976438 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55bce23-3a9a-4c6d-a9a1-ef51522a1190 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:14.989846 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-828defc5-f243-43a5-857a-a3ec41f0bb89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:14.989846 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:14.989846 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:14.989846 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:14.989846 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:14.989846 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:14.989846 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:14.989846 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:14.989846 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:14.989846 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:14.989846 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:14.989846 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:14.989846 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:30:14.989846 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:30:14.989846 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:30:14.989846 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:30:14.990824 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:14.990824 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:14.990824 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:14.990824 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:14.990824 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:14.990824 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:14.990824 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:14.990824 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:14.990824 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:14.990824 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:14.990824 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:14.990824 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:14.990824 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:14.990824 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:14.990824 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:14.990824 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:14.990824 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:30:14.991788 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:30:14.991788 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:14.991788 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:14.995950 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 395/795] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:30:14.997363 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b05bb-da4a-426c-a988-b01979e2432c None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:14.997691 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b05bb-da4a-426c-a988-b01979e2432c None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:14.998039 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b05bb-da4a-426c-a988-b01979e2432c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:15.027998 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:15.028847 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-a33b05bb-da4a-426c-a988-b01979e2432c None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:15.030737 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-a33b05bb-da4a-426c-a988-b01979e2432c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:15.076535 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:15.125058 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55bce23-3a9a-4c6d-a9a1-ef51522a1190 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:15.131294 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 396/796] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:30:15.131294 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b05bb-da4a-426c-a988-b01979e2432c None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:15.234129 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e82cb-6336-4e26-b817-5e43eacc8e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:15.235786 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e82cb-6336-4e26-b817-5e43eacc8e02 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:15.236883 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e82cb-6336-4e26-b817-5e43eacc8e02 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:15.237299 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e82cb-6336-4e26-b817-5e43eacc8e02 None admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:15.239460 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75e82cb-6336-4e26-b817-5e43eacc8e02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:15.239960 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75e82cb-6336-4e26-b817-5e43eacc8e02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:15.242344 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75e82cb-6336-4e26-b817-5e43eacc8e02 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:15.275290 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 401/797] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:30:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:15.276934 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e82cb-6336-4e26-b817-5e43eacc8e02 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:15.327324 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b05bb-da4a-426c-a988-b01979e2432c None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:15.399586 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f742ba-cc35-456e-88f0-63bc0d3ac96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:15.400465 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f742ba-cc35-456e-88f0-63bc0d3ac96d None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:15.402382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f742ba-cc35-456e-88f0-63bc0d3ac96d None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:15.403180 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f742ba-cc35-456e-88f0-63bc0d3ac96d None admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:15.406692 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f742ba-cc35-456e-88f0-63bc0d3ac96d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:15.407178 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f742ba-cc35-456e-88f0-63bc0d3ac96d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:15.408740 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f742ba-cc35-456e-88f0-63bc0d3ac96d None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:15.436127 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 397/798] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:30:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:15.441608 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f742ba-cc35-456e-88f0-63bc0d3ac96d None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:15.457312 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:15.466161 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e82cb-6336-4e26-b817-5e43eacc8e02 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:15.484819 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:15.509390 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84c366e-06af-4796-9d4e-9fcfab1a03d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:15.510554 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c366e-06af-4796-9d4e-9fcfab1a03d3 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:15.511032 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c366e-06af-4796-9d4e-9fcfab1a03d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:15.511528 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c366e-06af-4796-9d4e-9fcfab1a03d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:15.519127 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84c366e-06af-4796-9d4e-9fcfab1a03d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:15.520601 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84c366e-06af-4796-9d4e-9fcfab1a03d3 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:15.531615 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-b84c366e-06af-4796-9d4e-9fcfab1a03d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:30:15.531615 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:15.531615 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:15.531615 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:15.531615 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:15.531615 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:15.531615 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:15.531615 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:15.531615 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:15.531615 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:15.531615 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:15.531615 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:15.531615 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:30:15.531615 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:30:15.531615 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:30:15.532688 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:30:15.532688 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:15.532688 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:15.532688 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:15.532688 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:15.532688 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:15.532688 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:15.532688 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:15.532688 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:15.532688 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:15.532688 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:15.532688 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:15.532688 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:15.532688 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:15.532688 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:15.532688 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:15.532688 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:15.533592 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:30:15.533592 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:30:15.533592 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:30:15.533592 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:30:15.533592 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:30:15.533592 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:30:15.533592 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:30:15.533592 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:15.533592 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 402/799] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:30:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:15.534175 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84c366e-06af-4796-9d4e-9fcfab1a03d3 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:15.573879 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f742ba-cc35-456e-88f0-63bc0d3ac96d None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:15.608187 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ba364b-31c8-4c7b-b86b-f2d9e9f4832d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:15.608823 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba364b-31c8-4c7b-b86b-f2d9e9f4832d None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:15.609099 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba364b-31c8-4c7b-b86b-f2d9e9f4832d None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:15.609367 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ba364b-31c8-4c7b-b86b-f2d9e9f4832d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:15.620661 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ba364b-31c8-4c7b-b86b-f2d9e9f4832d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:15.625860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-19ba364b-31c8-4c7b-b86b-f2d9e9f4832d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:30:15.625860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:15.625860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:15.625860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:15.625860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:15.625860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:15.625860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:15.625860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:15.625860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:15.625860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:15.625860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:15.625860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:15.625860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:15.625860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:15.625860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:15.625860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:15.626779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:15.626779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:15.626779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:15.626779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:15.626779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:15.626779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:15.626779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:15.626779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:15.626779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:15.626779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:15.626779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:15.626779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:15.626779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:15.626779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:15.626779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:15.626779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:15.627645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:15.627645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:15.627645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:15.627645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:15.627645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:15.627645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:15.627645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:15.627645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:15.627645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:15.627645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:15.627645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:30:15.627645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:15.627645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 398/800] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 30 14:30:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:15.628906 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7dda4-8a57-48de-9719-3e4857c2f2ae None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:15.628906 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7dda4-8a57-48de-9719-3e4857c2f2ae None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:15.628906 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7dda4-8a57-48de-9719-3e4857c2f2ae None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:15.634344 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 399/801] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:15.634344 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b7dda4-8a57-48de-9719-3e4857c2f2ae None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:15.650232 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84c366e-06af-4796-9d4e-9fcfab1a03d3 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:15.691979 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebf1710-fe51-4ab6-8b30-89d4c575ea7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:15.692601 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebf1710-fe51-4ab6-8b30-89d4c575ea7f None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:15.692886 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebf1710-fe51-4ab6-8b30-89d4c575ea7f None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:15.693205 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebf1710-fe51-4ab6-8b30-89d4c575ea7f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:15.702122 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebf1710-fe51-4ab6-8b30-89d4c575ea7f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:15.703548 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebf1710-fe51-4ab6-8b30-89d4c575ea7f None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:15.716391 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-0ebf1710-fe51-4ab6-8b30-89d4c575ea7f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 14:30:15.716391 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:15.716391 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:15.716391 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:15.716391 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:15.716391 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:15.716391 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:15.716391 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:15.716391 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:15.716391 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:15.716391 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:15.716391 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:15.716391 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:30:15.716391 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:30:15.716391 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:30:15.716391 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:30:15.717573 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:15.717573 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:15.717573 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:30:15.717573 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:30:15.717573 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:30:15.717573 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:30:15.717573 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:15.717573 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:15.717573 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:15.717573 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:15.717573 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:15.717573 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:15.717573 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:15.717573 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:15.717573 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:15.717573 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:15.717573 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:15.717573 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:15.718823 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:15.718823 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:15.718823 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:30:15.718823 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:30:15.718823 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:30:15.718823 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:30:15.718823 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:30:15.718823 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:30:15.718823 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 14:30:15.718823 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:15.718823 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 403/802] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 30 14:30:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:15.718823 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebf1710-fe51-4ab6-8b30-89d4c575ea7f None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:15.740981 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:15.743769 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:15.760173 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b7dda4-8a57-48de-9719-3e4857c2f2ae None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:15.799850 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de7ec03-863a-4ca1-a398-ef5103fba494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:15.800526 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7ec03-863a-4ca1-a398-ef5103fba494 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:15.800813 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7ec03-863a-4ca1-a398-ef5103fba494 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:15.801085 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7ec03-863a-4ca1-a398-ef5103fba494 None admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:15.801775 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de7ec03-863a-4ca1-a398-ef5103fba494 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:15.802138 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de7ec03-863a-4ca1-a398-ef5103fba494 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:15.803592 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de7ec03-863a-4ca1-a398-ef5103fba494 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:15.819368 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 400/803] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 30 14:30:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:15.822752 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271425d-484c-4724-bdac-ddbe5fbfc758 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:15.823034 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271425d-484c-4724-bdac-ddbe5fbfc758 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:15.823344 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271425d-484c-4724-bdac-ddbe5fbfc758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:15.844341 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-9271425d-484c-4724-bdac-ddbe5fbfc758 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:15.845778 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-9271425d-484c-4724-bdac-ddbe5fbfc758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:15.855408 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebf1710-fe51-4ab6-8b30-89d4c575ea7f None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:15.897247 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c78b22-cefa-4126-b8be-6de37ad327f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:15.898098 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c78b22-cefa-4126-b8be-6de37ad327f7 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:15.898560 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c78b22-cefa-4126-b8be-6de37ad327f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:15.898827 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c78b22-cefa-4126-b8be-6de37ad327f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:15.899701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c78b22-cefa-4126-b8be-6de37ad327f7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:15.899992 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c78b22-cefa-4126-b8be-6de37ad327f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:15.901539 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c78b22-cefa-4126-b8be-6de37ad327f7 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:15.919431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 404/804] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:30:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:15.922480 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c78b22-cefa-4126-b8be-6de37ad327f7 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:15.947258 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 401/805] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:30:15.947258 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9271425d-484c-4724-bdac-ddbe5fbfc758 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:16.030046 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:16.045604 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:16.068862 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9271425d-484c-4724-bdac-ddbe5fbfc758 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:16.071042 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c78b22-cefa-4126-b8be-6de37ad327f7 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:16.106614 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c946a-721a-4fac-a3af-0a86b5bd620a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:16.107312 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c946a-721a-4fac-a3af-0a86b5bd620a None admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:16.107598 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c946a-721a-4fac-a3af-0a86b5bd620a None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:16.107881 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c946a-721a-4fac-a3af-0a86b5bd620a None admin] PATH_INFO: `/v3/projects/01cf71ca5abf4576b89d10c85af6d2de/users/f60bee471e954932a928ab2cf7d66c82/roles/1316db16eac54629a283545f1601a236` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:16.120457 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d705269-0369-4a34-8e78-9d0eb62ad93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:16.121039 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d705269-0369-4a34-8e78-9d0eb62ad93a None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:16.121408 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d705269-0369-4a34-8e78-9d0eb62ad93a None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:16.121768 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d705269-0369-4a34-8e78-9d0eb62ad93a None admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:16.122610 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d705269-0369-4a34-8e78-9d0eb62ad93a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:16.122975 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d705269-0369-4a34-8e78-9d0eb62ad93a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:16.124504 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d705269-0369-4a34-8e78-9d0eb62ad93a None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:16.125063 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c946a-721a-4fac-a3af-0a86b5bd620a None admin] RBAC: Authorizing `identity:create_grant(project_id=01cf71ca5abf4576b89d10c85af6d2de, user_id=f60bee471e954932a928ab2cf7d66c82, role_id=1316db16eac54629a283545f1601a236)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:16.126659 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c946a-721a-4fac-a3af-0a86b5bd620a None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:16.144311 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 402/806] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 30 14:30:15 2022] PUT /identity/v3/projects/01cf71ca5abf4576b89d10c85af6d2de/users/f60bee471e954932a928ab2cf7d66c82/roles/1316db16eac54629a283545f1601a236 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:30:16.145180 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c946a-721a-4fac-a3af-0a86b5bd620a None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:16.156668 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 405/807] 10.222.0.30 () {56 vars in 1135 bytes} [Tue Aug 30 14:30:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:16.158082 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d09c2f-f367-47d0-b454-6e41183d403c None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:16.158426 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d09c2f-f367-47d0-b454-6e41183d403c None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:16.158767 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d09c2f-f367-47d0-b454-6e41183d403c None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:16.171893 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 406/808] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:16 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:16.172955 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d09c2f-f367-47d0-b454-6e41183d403c None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:16.285569 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c946a-721a-4fac-a3af-0a86b5bd620a None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:16.335641 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:16.342908 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f7c305-5d1f-4405-8f2a-5c657298897a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:16.343799 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7c305-5d1f-4405-8f2a-5c657298897a None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:16.347233 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7c305-5d1f-4405-8f2a-5c657298897a None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:16.351230 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7c305-5d1f-4405-8f2a-5c657298897a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:16.367231 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f7c305-5d1f-4405-8f2a-5c657298897a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:16.367231 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f7c305-5d1f-4405-8f2a-5c657298897a None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:16.371239 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d09c2f-f367-47d0-b454-6e41183d403c None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:16.384916 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-58f7c305-5d1f-4405-8f2a-5c657298897a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:16.384916 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:16.384916 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:16.384916 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:16.384916 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:16.384916 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:16.384916 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:16.384916 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:16.384916 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:16.384916 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:16.384916 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:16.384916 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:16.384916 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:30:16.384916 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:30:16.384916 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:30:16.384916 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:30:16.386574 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:16.386574 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:16.386574 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:16.386574 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:16.386574 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:16.386574 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:16.386574 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:16.386574 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:16.386574 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:16.386574 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:16.386574 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:16.386574 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:16.386574 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:16.386574 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:16.386574 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:16.386574 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:16.386574 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:30:16.387625 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:30:16.387625 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:16.387625 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:16.387625 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 403/809] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:16.395502 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd60d70-7ba5-452c-a29c-d77ae2a71d43 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:16.395502 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd60d70-7ba5-452c-a29c-d77ae2a71d43 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:16.395502 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd60d70-7ba5-452c-a29c-d77ae2a71d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:16.419678 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe22dd0-e07b-4fa2-bcb6-23220141c296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:16.420490 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe22dd0-e07b-4fa2-bcb6-23220141c296 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:16.420836 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe22dd0-e07b-4fa2-bcb6-23220141c296 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:16.421212 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe22dd0-e07b-4fa2-bcb6-23220141c296 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:16.424483 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:16.428172 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-bfd60d70-7ba5-452c-a29c-d77ae2a71d43 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:16.430563 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-bfd60d70-7ba5-452c-a29c-d77ae2a71d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:16.441040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe22dd0-e07b-4fa2-bcb6-23220141c296 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:16.442676 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe22dd0-e07b-4fa2-bcb6-23220141c296 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:16.459929 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-6fe22dd0-e07b-4fa2-bcb6-23220141c296 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:16.459929 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:16.459929 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:16.459929 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:16.459929 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:16.459929 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:16.459929 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:16.459929 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:16.459929 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:16.459929 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:16.459929 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:16.459929 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:16.459929 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:30:16.459929 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:30:16.459929 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:30:16.459929 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:30:16.461000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:16.461000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:16.461000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:16.461000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:16.461000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:16.461000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:16.461000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:16.461000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:16.461000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:16.461000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:16.461000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:16.461000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:16.461000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:16.461000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:16.461000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:16.461000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:16.461000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:30:16.461907 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:30:16.461907 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:16.461907 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:16.463886 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 407/810] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:16.464820 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe22dd0-e07b-4fa2-bcb6-23220141c296 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:16.536096 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 404/811] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:30:16.537283 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd60d70-7ba5-452c-a29c-d77ae2a71d43 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:16.648478 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe22dd0-e07b-4fa2-bcb6-23220141c296 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:16.724496 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a203b0f-a797-449d-8c20-d6634893c7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:16.725629 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a203b0f-a797-449d-8c20-d6634893c7fb None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:16.726111 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a203b0f-a797-449d-8c20-d6634893c7fb None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:16.726580 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a203b0f-a797-449d-8c20-d6634893c7fb None admin] PATH_INFO: `/v3/services/network` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:16.727405 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a203b0f-a797-449d-8c20-d6634893c7fb None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:16.729595 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a203b0f-a797-449d-8c20-d6634893c7fb None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:16.749483 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-0a203b0f-a797-449d-8c20-d6634893c7fb None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 14:30:16.749483 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:16.749483 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:30:16.749483 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:30:16.749483 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:30:16.749483 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:30:16.749483 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:30:16.749483 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:30:16.749483 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 14:30:16.749483 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:16.749483 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:30:16.749483 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:16.749483 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:16.749483 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:16.749483 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:16.749483 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:16.749483 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:16.750505 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:16.750505 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:16.750505 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:16.750505 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:16.750505 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:16.750505 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:16.750505 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:30:16.750505 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:30:16.750505 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:30:16.750505 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:30:16.750505 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:16.750505 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:16.750505 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:16.750505 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:16.750505 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:16.750505 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:16.751377 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:16.751377 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:16.751377 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:16.751377 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:16.751377 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:16.751377 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:16.751377 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:16.751377 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:16.751377 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:16.751377 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:16.751377 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:30:16.751377 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:30:16.751377 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 14:30:16.751377 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:16.753002 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 408/812] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:30:16 2022] GET /identity/v3/services/network => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:16.753802 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a203b0f-a797-449d-8c20-d6634893c7fb None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:16.759386 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd60d70-7ba5-452c-a29c-d77ae2a71d43 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:16.780743 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:16.821289 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a092408-0b8f-4004-afda-50c2e5e283c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:16.821903 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a092408-0b8f-4004-afda-50c2e5e283c5 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:16.822182 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a092408-0b8f-4004-afda-50c2e5e283c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:16.822452 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a092408-0b8f-4004-afda-50c2e5e283c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:16.823703 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a092408-0b8f-4004-afda-50c2e5e283c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:16.824101 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a092408-0b8f-4004-afda-50c2e5e283c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:16.825469 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a092408-0b8f-4004-afda-50c2e5e283c5 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:16.845381 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 405/813] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:30:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:16.845575 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a092408-0b8f-4004-afda-50c2e5e283c5 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:16.865782 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:16.921786 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a203b0f-a797-449d-8c20-d6634893c7fb None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:16.969158 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bcaf94-1c41-471d-b86f-865e225974ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:16.969764 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bcaf94-1c41-471d-b86f-865e225974ec None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:16.970043 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bcaf94-1c41-471d-b86f-865e225974ec None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:16.970309 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bcaf94-1c41-471d-b86f-865e225974ec None admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:16.970946 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bcaf94-1c41-471d-b86f-865e225974ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:16.971284 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bcaf94-1c41-471d-b86f-865e225974ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:16.972646 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bcaf94-1c41-471d-b86f-865e225974ec None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:16.986122 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 409/814] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:30:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:16.986903 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bcaf94-1c41-471d-b86f-865e225974ec None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:16.999631 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a092408-0b8f-4004-afda-50c2e5e283c5 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:17.039658 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4ec624-dcdf-487d-af13-c2f5d8d72082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:17.040424 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ec624-dcdf-487d-af13-c2f5d8d72082 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:17.040705 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ec624-dcdf-487d-af13-c2f5d8d72082 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:17.040979 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ec624-dcdf-487d-af13-c2f5d8d72082 None admin] PATH_INFO: `/v3/services` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:17.042318 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4ec624-dcdf-487d-af13-c2f5d8d72082 None admin] RBAC: Adding query filter params (name=network) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:17.042771 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4ec624-dcdf-487d-af13-c2f5d8d72082 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:17.044191 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4ec624-dcdf-487d-af13-c2f5d8d72082 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:17.053225 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 406/815] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Aug 30 14:30:16 2022] GET /identity/v3/services?name=network => generated 124 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:17.054057 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4ec624-dcdf-487d-af13-c2f5d8d72082 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:17.100151 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:17.159234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bcaf94-1c41-471d-b86f-865e225974ec None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:17.175668 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:17.179000 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4ec624-dcdf-487d-af13-c2f5d8d72082 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:17.219966 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae5707-91f8-4971-ad2c-c6d448091d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:17.220628 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae5707-91f8-4971-ad2c-c6d448091d54 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:17.220871 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae5707-91f8-4971-ad2c-c6d448091d54 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:17.221142 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae5707-91f8-4971-ad2c-c6d448091d54 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:17.221841 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ae5707-91f8-4971-ad2c-c6d448091d54 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:17.223232 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ae5707-91f8-4971-ad2c-c6d448091d54 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:17.224698 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd41720-3857-434f-8d4a-326d1163b6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:17.225621 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd41720-3857-434f-8d4a-326d1163b6ac None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:17.225997 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd41720-3857-434f-8d4a-326d1163b6ac None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:17.226361 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd41720-3857-434f-8d4a-326d1163b6ac None admin] PATH_INFO: `/v3/users/nova` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:17.237612 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd41720-3857-434f-8d4a-326d1163b6ac None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:17.241073 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-66ae5707-91f8-4971-ad2c-c6d448091d54 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 14:30:17.241073 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:17.241073 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:30:17.241073 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:30:17.241073 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:30:17.241073 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:30:17.241073 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:30:17.241073 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:30:17.241073 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 14:30:17.241073 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:17.241073 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:30:17.241073 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:17.241073 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:17.241073 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:17.241073 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:17.241073 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:17.241073 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:17.242162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:17.242162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:17.242162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:17.242162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:17.242162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:17.242162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:17.242162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:30:17.242162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:30:17.242162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:30:17.242162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:30:17.242162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:17.242162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:17.242162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:17.242162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:17.242162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:17.242162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:17.243040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:17.243040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:17.243040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:17.243040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:17.243040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:17.243040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:17.243040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:17.243040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:17.243040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:17.243040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:17.243040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:30:17.243040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:30:17.243040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 14:30:17.243040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:17.243846 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd41720-3857-434f-8d4a-326d1163b6ac None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:17.243927 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 410/816] 10.222.0.30 () {56 vars in 1129 bytes} [Tue Aug 30 14:30:16 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:17.243927 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae5707-91f8-4971-ad2c-c6d448091d54 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:17.259253 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-5fd41720-3857-434f-8d4a-326d1163b6ac None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:30:17.259253 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:17.259253 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:17.259253 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:17.259253 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:17.259253 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:17.259253 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:17.259253 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:17.259253 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:17.259253 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:17.259253 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:17.259253 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:17.259253 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:30:17.259253 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:30:17.259253 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:30:17.259253 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:30:17.261006 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:17.261006 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:17.261006 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:30:17.261006 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:30:17.261006 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:30:17.261006 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:30:17.261006 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:17.261006 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:17.261006 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:17.261006 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:17.261006 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:17.261006 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:17.261006 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:17.261006 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:17.261006 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:17.261006 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:17.261006 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:17.261006 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:17.262055 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:17.262055 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:17.262055 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:30:17.262055 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:30:17.262055 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:30:17.262055 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:30:17.262055 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:30:17.262055 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:30:17.262055 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:30:17.262055 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:17.267247 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 407/817] 10.222.0.30 () {56 vars in 1093 bytes} [Tue Aug 30 14:30:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:17.267247 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd41720-3857-434f-8d4a-326d1163b6ac None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:17.409105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae5707-91f8-4971-ad2c-c6d448091d54 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:17.419466 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:17.430041 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd41720-3857-434f-8d4a-326d1163b6ac None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:17.466443 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44959237-1585-4bfe-92c6-a45860de0177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:17.467095 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44959237-1585-4bfe-92c6-a45860de0177 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:17.467406 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44959237-1585-4bfe-92c6-a45860de0177 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:17.467679 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44959237-1585-4bfe-92c6-a45860de0177 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:17.476420 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44959237-1585-4bfe-92c6-a45860de0177 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:17.478037 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44959237-1585-4bfe-92c6-a45860de0177 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:17.490579 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-44959237-1585-4bfe-92c6-a45860de0177 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:30:17.490579 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:17.490579 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:17.490579 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:17.490579 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:17.490579 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:17.490579 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:17.490579 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:17.490579 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:17.490579 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:17.490579 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:17.490579 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:17.490579 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:30:17.490579 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:30:17.490579 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:30:17.491766 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:30:17.491766 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:17.491766 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:17.491766 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:17.491766 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:17.491766 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:17.491766 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:17.491766 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:17.491766 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:17.491766 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:17.491766 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:17.491766 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:17.491766 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:17.491766 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:17.491766 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:17.491766 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:17.491766 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:17.492819 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:30:17.492819 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:30:17.492819 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:30:17.492819 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:30:17.492819 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:30:17.492819 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:30:17.492819 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:30:17.492819 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:17.493661 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee97951-de7d-4a8f-957b-36f7997c85f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:17.494284 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee97951-de7d-4a8f-957b-36f7997c85f1 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:17.495093 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee97951-de7d-4a8f-957b-36f7997c85f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:17.495419 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee97951-de7d-4a8f-957b-36f7997c85f1 None admin] PATH_INFO: `/v3/services` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:17.496026 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 411/818] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:30:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:17.497046 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44959237-1585-4bfe-92c6-a45860de0177 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:17.499787 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee97951-de7d-4a8f-957b-36f7997c85f1 None admin] RBAC: Adding query filter params (type=network) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:17.500257 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee97951-de7d-4a8f-957b-36f7997c85f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:17.502090 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee97951-de7d-4a8f-957b-36f7997c85f1 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:17.513403 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 408/819] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Aug 30 14:30:17 2022] GET /identity/v3/services?type=network => generated 350 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:17.514633 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee97951-de7d-4a8f-957b-36f7997c85f1 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:17.563605 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:17.650755 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee97951-de7d-4a8f-957b-36f7997c85f1 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:17.692565 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107099d7-a66b-4a87-b23d-c145d1f35a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:17.693235 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107099d7-a66b-4a87-b23d-c145d1f35a94 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:17.693610 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107099d7-a66b-4a87-b23d-c145d1f35a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:17.693973 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107099d7-a66b-4a87-b23d-c145d1f35a94 None admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:17.694799 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107099d7-a66b-4a87-b23d-c145d1f35a94 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:17.695196 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107099d7-a66b-4a87-b23d-c145d1f35a94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:17.696718 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107099d7-a66b-4a87-b23d-c145d1f35a94 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:17.698939 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44959237-1585-4bfe-92c6-a45860de0177 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:17.720644 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 409/820] 10.222.0.30 () {56 vars in 1133 bytes} [Tue Aug 30 14:30:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:17.721582 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107099d7-a66b-4a87-b23d-c145d1f35a94 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:17.741840 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:17.746236 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ee5c51-2dda-4512-8da8-f4dd0feae10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:17.746925 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee5c51-2dda-4512-8da8-f4dd0feae10e None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:17.747332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee5c51-2dda-4512-8da8-f4dd0feae10e None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:17.747747 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee5c51-2dda-4512-8da8-f4dd0feae10e None admin] PATH_INFO: `/v3/services` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:17.748350 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ee5c51-2dda-4512-8da8-f4dd0feae10e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:17.748715 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ee5c51-2dda-4512-8da8-f4dd0feae10e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:17.750145 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ee5c51-2dda-4512-8da8-f4dd0feae10e None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:17.755120 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 412/821] 10.222.0.30 () {56 vars in 1124 bytes} [Tue Aug 30 14:30:17 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:17.756010 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ee5c51-2dda-4512-8da8-f4dd0feae10e None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:17.843654 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107099d7-a66b-4a87-b23d-c145d1f35a94 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:17.871555 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:17.875427 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ee5c51-2dda-4512-8da8-f4dd0feae10e None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:17.881425 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cd61f0-d519-4ba3-83c8-ebe3d43e803d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:17.882037 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd61f0-d519-4ba3-83c8-ebe3d43e803d None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:17.882317 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd61f0-d519-4ba3-83c8-ebe3d43e803d None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:17.882589 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd61f0-d519-4ba3-83c8-ebe3d43e803d None admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:17.883306 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cd61f0-d519-4ba3-83c8-ebe3d43e803d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:17.883615 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cd61f0-d519-4ba3-83c8-ebe3d43e803d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:17.885108 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cd61f0-d519-4ba3-83c8-ebe3d43e803d None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:17.897170 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 410/822] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 30 14:30:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:17.898267 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cd61f0-d519-4ba3-83c8-ebe3d43e803d None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:17.918402 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6209f2-ff06-4d79-a844-431879f9a122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:17.919015 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6209f2-ff06-4d79-a844-431879f9a122 None admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:17.919323 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6209f2-ff06-4d79-a844-431879f9a122 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:17.919597 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6209f2-ff06-4d79-a844-431879f9a122 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:17.920268 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6209f2-ff06-4d79-a844-431879f9a122 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:17.921627 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6209f2-ff06-4d79-a844-431879f9a122 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:17.944854 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 413/823] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:30:17 2022] POST /identity/v3/endpoints => generated 337 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:30:17.945662 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6209f2-ff06-4d79-a844-431879f9a122 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:18.004264 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:18.039550 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cd61f0-d519-4ba3-83c8-ebe3d43e803d None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:18.085260 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb2a22b-6bc5-4a33-84cf-6ef9dae1fc32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:18.086186 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb2a22b-6bc5-4a33-84cf-6ef9dae1fc32 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:18.086690 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb2a22b-6bc5-4a33-84cf-6ef9dae1fc32 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:18.087270 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb2a22b-6bc5-4a33-84cf-6ef9dae1fc32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:18.106090 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6209f2-ff06-4d79-a844-431879f9a122 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:18.152151 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760e6b8b-aacc-4291-9ec1-9c6cca49a06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:18.152785 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760e6b8b-aacc-4291-9ec1-9c6cca49a06a None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:18.153067 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760e6b8b-aacc-4291-9ec1-9c6cca49a06a None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:18.153341 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760e6b8b-aacc-4291-9ec1-9c6cca49a06a None admin] PATH_INFO: `/v3/services` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:18.153973 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760e6b8b-aacc-4291-9ec1-9c6cca49a06a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:18.154273 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760e6b8b-aacc-4291-9ec1-9c6cca49a06a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:18.155687 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760e6b8b-aacc-4291-9ec1-9c6cca49a06a None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:18.160889 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 414/824] 10.222.0.30 () {56 vars in 1124 bytes} [Tue Aug 30 14:30:17 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:18.161650 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760e6b8b-aacc-4291-9ec1-9c6cca49a06a None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:18.176561 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb2a22b-6bc5-4a33-84cf-6ef9dae1fc32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:18.178023 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb2a22b-6bc5-4a33-84cf-6ef9dae1fc32 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:18.187488 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:18.255566 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:18.269428 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760e6b8b-aacc-4291-9ec1-9c6cca49a06a None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:18.282889 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 411/825] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:30:17 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:30:18.285381 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40dc49d-a80c-437d-a3bc-b39112b32afc None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:18.285887 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40dc49d-a80c-437d-a3bc-b39112b32afc None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:18.286393 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40dc49d-a80c-437d-a3bc-b39112b32afc None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:18.289263 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 412/826] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:18.290920 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40dc49d-a80c-437d-a3bc-b39112b32afc None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:18.306322 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a4fdc-acdc-4c45-8461-a18b40375882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:18.307573 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a4fdc-acdc-4c45-8461-a18b40375882 None admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:18.308036 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a4fdc-acdc-4c45-8461-a18b40375882 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:18.308392 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a4fdc-acdc-4c45-8461-a18b40375882 None admin] PATH_INFO: `/v3/projects/01cf71ca5abf4576b89d10c85af6d2de/users/a18b13e3708144e1b5e8df80ec3a4e83/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:18.310604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104a4fdc-acdc-4c45-8461-a18b40375882 None admin] RBAC: Authorizing `identity:create_grant(project_id=01cf71ca5abf4576b89d10c85af6d2de, user_id=a18b13e3708144e1b5e8df80ec3a4e83, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:18.312087 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104a4fdc-acdc-4c45-8461-a18b40375882 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:18.325701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 415/827] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 30 14:30:18 2022] PUT /identity/v3/projects/01cf71ca5abf4576b89d10c85af6d2de/users/a18b13e3708144e1b5e8df80ec3a4e83/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:30:18.326494 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a4fdc-acdc-4c45-8461-a18b40375882 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:18.417057 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40dc49d-a80c-437d-a3bc-b39112b32afc None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:18.426564 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:18.449246 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a4fdc-acdc-4c45-8461-a18b40375882 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:18.470889 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae8f2ee-e762-486b-8d55-c1276a46b7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:18.470889 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae8f2ee-e762-486b-8d55-c1276a46b7fd None admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:18.471156 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae8f2ee-e762-486b-8d55-c1276a46b7fd None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:18.471156 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae8f2ee-e762-486b-8d55-c1276a46b7fd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:18.471873 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae8f2ee-e762-486b-8d55-c1276a46b7fd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:18.473249 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae8f2ee-e762-486b-8d55-c1276a46b7fd None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:18.492070 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c1d829-c042-4376-a589-8db74e33729d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:18.492676 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c1d829-c042-4376-a589-8db74e33729d None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:18.493040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c1d829-c042-4376-a589-8db74e33729d None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:18.493279 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c1d829-c042-4376-a589-8db74e33729d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:18.498967 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 413/828] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:30:18 2022] POST /identity/v3/endpoints => generated 346 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:30:18.500604 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e456c-c58b-4668-ad81-9de3098b968c None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:18.501081 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e456c-c58b-4668-ad81-9de3098b968c None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:18.501373 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e456c-c58b-4668-ad81-9de3098b968c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:18.505265 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c1d829-c042-4376-a589-8db74e33729d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:18.507167 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c1d829-c042-4376-a589-8db74e33729d None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:18.522220 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-178e456c-c58b-4668-ad81-9de3098b968c None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:18.523540 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-178e456c-c58b-4668-ad81-9de3098b968c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:18.529040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-b8c1d829-c042-4376-a589-8db74e33729d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:18.529040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:18.529040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:18.529040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:18.529040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:18.529040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:18.529040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:18.529040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:18.529040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:18.529040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:18.529040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:18.529040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:18.529040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:30:18.529040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:30:18.529040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:30:18.529040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:30:18.530105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:18.530105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:18.530105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:18.530105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:18.530105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:18.530105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:18.530105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:18.530105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:18.530105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:18.530105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:18.530105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:18.530105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:18.530105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:18.530105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:18.530105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:18.530105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:18.530105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:30:18.531010 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:30:18.531010 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:18.531010 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:18.533734 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 416/829] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:18.540005 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c1d829-c042-4376-a589-8db74e33729d None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:18.552321 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:18.617061 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 414/830] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:18 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:30:18.680697 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c1d829-c042-4376-a589-8db74e33729d None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:18.718360 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:18.727053 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fde7570-fb8e-44b3-aa97-686e8c566131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:18.727751 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde7570-fb8e-44b3-aa97-686e8c566131 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:18.728088 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde7570-fb8e-44b3-aa97-686e8c566131 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:18.728432 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde7570-fb8e-44b3-aa97-686e8c566131 None admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:18.729381 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fde7570-fb8e-44b3-aa97-686e8c566131 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:18.729755 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fde7570-fb8e-44b3-aa97-686e8c566131 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:18.731195 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fde7570-fb8e-44b3-aa97-686e8c566131 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:18.744210 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 417/831] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:30:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:30:18.749707 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e456c-c58b-4668-ad81-9de3098b968c None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:18.867338 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e456c-c58b-4668-ad81-9de3098b968c None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:18.887552 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fde7570-fb8e-44b3-aa97-686e8c566131 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:18.917633 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a167cbe-ff51-467f-b105-c1625ac30bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:18.918889 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a167cbe-ff51-467f-b105-c1625ac30bcb None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:18.919847 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a167cbe-ff51-467f-b105-c1625ac30bcb None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:18.920597 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a167cbe-ff51-467f-b105-c1625ac30bcb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:18.931045 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a167cbe-ff51-467f-b105-c1625ac30bcb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:18.933349 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a167cbe-ff51-467f-b105-c1625ac30bcb None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:18.949812 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-7a167cbe-ff51-467f-b105-c1625ac30bcb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:30:18.949812 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:18.949812 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:18.949812 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:18.949812 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:18.949812 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:18.949812 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:18.949812 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:18.949812 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:18.949812 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:18.949812 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:18.949812 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:18.949812 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:30:18.949812 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:30:18.949812 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:30:18.950769 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:30:18.950769 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:18.950769 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:18.950769 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:18.950769 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:18.950769 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:18.950769 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:18.950769 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:18.950769 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:18.950769 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:18.950769 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:18.950769 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:18.950769 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:18.950769 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:18.950769 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:18.950769 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:18.950769 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:18.951709 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:30:18.951709 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:30:18.951709 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:30:18.951709 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:30:18.951709 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:30:18.951709 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:30:18.951709 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:30:18.951709 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:18.957680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 415/832] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:30:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:18.958630 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a167cbe-ff51-467f-b105-c1625ac30bcb None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:18.984055 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:19.019078 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fde7570-fb8e-44b3-aa97-686e8c566131 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:19.063792 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d756865-bcc9-4df7-83a4-402502240b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:19.064428 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d756865-bcc9-4df7-83a4-402502240b82 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:19.064702 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d756865-bcc9-4df7-83a4-402502240b82 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:19.064969 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d756865-bcc9-4df7-83a4-402502240b82 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:19.077727 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d756865-bcc9-4df7-83a4-402502240b82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:19.083241 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-8d756865-bcc9-4df7-83a4-402502240b82 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:30:19.083241 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:19.083241 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:19.083241 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:19.083241 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:19.083241 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:19.083241 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:19.083241 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:19.083241 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:19.083241 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:19.083241 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:19.083241 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:19.083241 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:19.083241 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:19.083241 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:19.083241 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:19.084165 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:19.084165 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:19.084165 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:19.084165 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:19.084165 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:19.084165 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:19.084165 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:19.084165 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:19.084165 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:19.084165 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:19.084165 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:19.084165 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:19.084165 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:19.084165 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:19.084165 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:19.084165 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:19.085035 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:19.085035 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:19.085035 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:19.085035 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:19.085035 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:19.085035 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:19.085035 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:19.085035 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:19.085035 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:19.085035 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:19.085035 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:30:19.085035 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:19.087360 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 418/833] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 30 14:30:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:30:19.091813 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d756865-bcc9-4df7-83a4-402502240b82 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:19.106369 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a167cbe-ff51-467f-b105-c1625ac30bcb None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:19.144648 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4a9c9e-9ab9-4285-8777-027763f79b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:19.145433 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4a9c9e-9ab9-4285-8777-027763f79b5b None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:19.145633 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4a9c9e-9ab9-4285-8777-027763f79b5b None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:19.145888 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4a9c9e-9ab9-4285-8777-027763f79b5b None admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:19.146624 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4a9c9e-9ab9-4285-8777-027763f79b5b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:19.146928 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4a9c9e-9ab9-4285-8777-027763f79b5b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:19.148370 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4a9c9e-9ab9-4285-8777-027763f79b5b None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:19.159085 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:19.166920 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 416/834] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 30 14:30:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:30:19.169315 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4a9c9e-9ab9-4285-8777-027763f79b5b None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:19.221679 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d756865-bcc9-4df7-83a4-402502240b82 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:19.255725 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:19.262965 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eb99b8-9f90-4bec-a717-122b3be4184e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:19.263966 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb99b8-9f90-4bec-a717-122b3be4184e None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:19.264452 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb99b8-9f90-4bec-a717-122b3be4184e None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:19.264997 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb99b8-9f90-4bec-a717-122b3be4184e None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:19.265857 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eb99b8-9f90-4bec-a717-122b3be4184e None admin] RBAC: Adding query filter params (name=service) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:19.266362 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eb99b8-9f90-4bec-a717-122b3be4184e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:19.268246 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eb99b8-9f90-4bec-a717-122b3be4184e None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:19.278443 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 419/835] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 30 14:30:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:19.284504 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eb99b8-9f90-4bec-a717-122b3be4184e None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:19.284824 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4a9c9e-9ab9-4285-8777-027763f79b5b None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:19.322263 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d573342d-8de5-4088-a03d-616ff8727a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:19.323532 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d573342d-8de5-4088-a03d-616ff8727a2e None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:19.323808 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d573342d-8de5-4088-a03d-616ff8727a2e None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:19.324071 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d573342d-8de5-4088-a03d-616ff8727a2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:19.324788 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d573342d-8de5-4088-a03d-616ff8727a2e None admin] RBAC: Adding query filter params (role.id=3cda0615d87d4af0860a2a5e9b4e2c05, scope.project.id=01cf71ca5abf4576b89d10c85af6d2de, user.id=8878de66a03044e38e507028853a0cd1) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:19.325089 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d573342d-8de5-4088-a03d-616ff8727a2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:19.326511 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d573342d-8de5-4088-a03d-616ff8727a2e None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:19.341166 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 417/836] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:30:19 2022] GET /identity/v3/role_assignments?user.id=8878de66a03044e38e507028853a0cd1&scope.project.id=01cf71ca5abf4576b89d10c85af6d2de&role.id=3cda0615d87d4af0860a2a5e9b4e2c05 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:19.420044 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eb99b8-9f90-4bec-a717-122b3be4184e None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:19.437084 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:19.461119 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc07f950-29f9-4441-ad85-ee7e35f25ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:19.462090 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc07f950-29f9-4441-ad85-ee7e35f25ca7 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:19.462523 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc07f950-29f9-4441-ad85-ee7e35f25ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:19.463017 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc07f950-29f9-4441-ad85-ee7e35f25ca7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:19.481955 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc07f950-29f9-4441-ad85-ee7e35f25ca7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:19.483773 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc07f950-29f9-4441-ad85-ee7e35f25ca7 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:19.500057 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-bc07f950-29f9-4441-ad85-ee7e35f25ca7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:19.500057 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:19.500057 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:19.500057 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:19.500057 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:19.500057 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:19.500057 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:19.500057 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:19.500057 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:19.500057 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:19.500057 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:19.500057 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:19.500057 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:30:19.500057 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:30:19.500057 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:30:19.500057 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:30:19.501267 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:19.501267 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:19.501267 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:19.501267 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:19.501267 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:19.501267 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:19.501267 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:19.501267 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:19.501267 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:19.501267 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:19.501267 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:19.501267 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:19.501267 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:19.501267 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:19.501267 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:19.501267 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:19.501267 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:30:19.502383 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:30:19.502383 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:19.502383 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:19.504103 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 420/837] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:19.509738 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d573342d-8de5-4088-a03d-616ff8727a2e None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:19.615852 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d573342d-8de5-4088-a03d-616ff8727a2e None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:19.651634 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eac4fc-816d-4826-9918-73b4fbc8d910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:19.652362 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eac4fc-816d-4826-9918-73b4fbc8d910 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:19.652705 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eac4fc-816d-4826-9918-73b4fbc8d910 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:19.653044 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eac4fc-816d-4826-9918-73b4fbc8d910 None admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:19.653765 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eac4fc-816d-4826-9918-73b4fbc8d910 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:19.654128 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eac4fc-816d-4826-9918-73b4fbc8d910 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:19.655579 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eac4fc-816d-4826-9918-73b4fbc8d910 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:19.664084 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:19.668438 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 418/838] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:30:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:19.674235 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc07f950-29f9-4441-ad85-ee7e35f25ca7 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:19.785271 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc07f950-29f9-4441-ad85-ee7e35f25ca7 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:19.821147 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75ef6e7-6c36-45ad-9868-0a984a130b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:19.822140 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ef6e7-6c36-45ad-9868-0a984a130b48 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:19.822755 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ef6e7-6c36-45ad-9868-0a984a130b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:19.823254 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ef6e7-6c36-45ad-9868-0a984a130b48 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:19.831409 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75ef6e7-6c36-45ad-9868-0a984a130b48 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:19.833130 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75ef6e7-6c36-45ad-9868-0a984a130b48 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:19.845498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-e75ef6e7-6c36-45ad-9868-0a984a130b48 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 14:30:19.845498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:19.845498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:19.845498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:19.845498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:19.845498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:19.845498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:19.845498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:19.845498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:19.845498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:19.845498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:19.845498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:19.845498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:30:19.845498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:30:19.845498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:30:19.845498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:30:19.846450 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:19.846450 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:19.846450 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:30:19.846450 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:30:19.846450 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:30:19.846450 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:30:19.846450 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:19.846450 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:19.846450 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:19.846450 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:19.846450 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:19.846450 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:19.846450 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:19.846450 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:19.846450 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:19.846450 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:19.846450 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:19.846450 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:19.847491 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:19.847491 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:19.847491 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:30:19.847491 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:30:19.847491 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:30:19.847491 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:30:19.847491 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:30:19.847491 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:30:19.847491 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 14:30:19.847491 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:19.850029 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 421/839] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Aug 30 14:30:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:19.856457 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eac4fc-816d-4826-9918-73b4fbc8d910 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:19.970602 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eac4fc-816d-4826-9918-73b4fbc8d910 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:20.008830 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fc7764-749d-4d74-adb6-c7f7839f4bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:20.009796 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc7764-749d-4d74-adb6-c7f7839f4bae None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:20.010858 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc7764-749d-4d74-adb6-c7f7839f4bae None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:20.011502 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc7764-749d-4d74-adb6-c7f7839f4bae None admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:20.012472 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fc7764-749d-4d74-adb6-c7f7839f4bae None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:20.012984 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fc7764-749d-4d74-adb6-c7f7839f4bae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:20.014672 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fc7764-749d-4d74-adb6-c7f7839f4bae None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:20.036856 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 419/840] 10.222.0.30 () {56 vars in 1149 bytes} [Tue Aug 30 14:30:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:20.043034 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75ef6e7-6c36-45ad-9868-0a984a130b48 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:20.079838 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:20.157723 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75ef6e7-6c36-45ad-9868-0a984a130b48 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:20.192639 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7495d17-485d-45e1-b510-31c17ef92d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:20.193518 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7495d17-485d-45e1-b510-31c17ef92d16 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:20.193937 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7495d17-485d-45e1-b510-31c17ef92d16 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:20.194346 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7495d17-485d-45e1-b510-31c17ef92d16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:20.272135 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7495d17-485d-45e1-b510-31c17ef92d16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:20.273577 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7495d17-485d-45e1-b510-31c17ef92d16 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:20.301977 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:20.380946 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825690b4-538b-4cc3-92d7-7a444e7e033a None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:20.381698 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825690b4-538b-4cc3-92d7-7a444e7e033a None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:20.381916 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 422/841] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:30:20 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 14:30:20.382378 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825690b4-538b-4cc3-92d7-7a444e7e033a None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:20.384021 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 420/842] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:30:20.387796 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7495d17-485d-45e1-b510-31c17ef92d16 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:20.389990 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43add302-27cd-4234-93a4-b049735d9a81 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:20.390350 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43add302-27cd-4234-93a4-b049735d9a81 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:20.390720 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43add302-27cd-4234-93a4-b049735d9a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:20.409661 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-43add302-27cd-4234-93a4-b049735d9a81 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:20.411050 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-43add302-27cd-4234-93a4-b049735d9a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:20.480928 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:20.496174 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7495d17-485d-45e1-b510-31c17ef92d16 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:20.505723 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 421/843] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:20 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:30:20.527886 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:20.535554 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0307ce63-def2-495e-90fa-b9e6e1bc7689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:20.536246 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307ce63-def2-495e-90fa-b9e6e1bc7689 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:20.536577 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307ce63-def2-495e-90fa-b9e6e1bc7689 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:20.536855 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307ce63-def2-495e-90fa-b9e6e1bc7689 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:20.547327 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0307ce63-def2-495e-90fa-b9e6e1bc7689 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:20.548783 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0307ce63-def2-495e-90fa-b9e6e1bc7689 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:20.565893 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ad5ee-a0c8-4334-b564-a1f9f4d9f5e7 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:20.566193 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ad5ee-a0c8-4334-b564-a1f9f4d9f5e7 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:20.568457 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ad5ee-a0c8-4334-b564-a1f9f4d9f5e7 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:20.568567 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-0307ce63-def2-495e-90fa-b9e6e1bc7689 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:20.568567 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:20.568567 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:20.568567 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:20.568567 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:20.568567 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:20.568567 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:20.568567 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:20.568567 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:20.568567 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:20.568567 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:20.568567 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:20.568567 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:30:20.568567 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:30:20.568567 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:30:20.568567 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:30:20.569454 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:20.569454 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:20.569454 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:20.569454 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:20.569454 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:20.569454 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:20.569454 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:20.569454 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:20.569454 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:20.569454 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:20.569454 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:20.569454 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:20.569454 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:20.569454 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:20.569454 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:20.569454 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:20.569454 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:30:20.570342 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:30:20.570342 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:20.570342 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:20.570510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 422/844] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:30:20.570924 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 423/845] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:30:20.574259 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7ad5ee-a0c8-4334-b564-a1f9f4d9f5e7 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:20.577380 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f57cde-9345-4dc2-afc6-8f86f160c9d4 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:20.577882 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f57cde-9345-4dc2-afc6-8f86f160c9d4 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:20.578383 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f57cde-9345-4dc2-afc6-8f86f160c9d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:20.609171 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-42f57cde-9345-4dc2-afc6-8f86f160c9d4 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:20.610911 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-42f57cde-9345-4dc2-afc6-8f86f160c9d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:20.693161 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 424/846] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:20 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:30:20.698204 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7ad5ee-a0c8-4334-b564-a1f9f4d9f5e7 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:20.735157 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffc5f1e-2d6a-48ab-ad1c-efea263e5f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:20.735795 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc5f1e-2d6a-48ab-ad1c-efea263e5f95 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:20.736078 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc5f1e-2d6a-48ab-ad1c-efea263e5f95 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:20.736356 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc5f1e-2d6a-48ab-ad1c-efea263e5f95 None admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:20.736998 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffc5f1e-2d6a-48ab-ad1c-efea263e5f95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:20.737299 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffc5f1e-2d6a-48ab-ad1c-efea263e5f95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:20.738670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffc5f1e-2d6a-48ab-ad1c-efea263e5f95 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:20.755306 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 423/847] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:30:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:20.756715 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f57cde-9345-4dc2-afc6-8f86f160c9d4 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:20.791935 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffc5f1e-2d6a-48ab-ad1c-efea263e5f95 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:20.849470 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:20.892652 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f57cde-9345-4dc2-afc6-8f86f160c9d4 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:20.911286 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:20.915268 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffc5f1e-2d6a-48ab-ad1c-efea263e5f95 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:20.950068 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8de1527-1ede-4883-beba-2bcf5712c557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:20.951637 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de1527-1ede-4883-beba-2bcf5712c557 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:20.952163 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de1527-1ede-4883-beba-2bcf5712c557 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:20.952719 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de1527-1ede-4883-beba-2bcf5712c557 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:20.955674 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f45509-1086-479a-9006-9c0654c41e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:20.956563 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f45509-1086-479a-9006-9c0654c41e4a None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:20.956936 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f45509-1086-479a-9006-9c0654c41e4a None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:20.957307 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f45509-1086-479a-9006-9c0654c41e4a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:20.962503 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8de1527-1ede-4883-beba-2bcf5712c557 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:20.964873 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8de1527-1ede-4883-beba-2bcf5712c557 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:20.971466 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f45509-1086-479a-9006-9c0654c41e4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:20.977468 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-48f45509-1086-479a-9006-9c0654c41e4a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:30:20.977468 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:20.977468 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:20.977468 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:20.977468 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:20.977468 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:20.977468 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:20.977468 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:20.977468 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:20.977468 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:20.977468 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:20.977468 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:20.977468 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:20.977468 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:20.977468 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:20.977468 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:20.978385 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:20.978385 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:20.978385 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:20.978385 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:20.978385 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:20.978385 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:20.978385 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:20.978385 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:20.978385 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:20.978385 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:20.978385 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:20.978385 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:20.978385 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:20.978385 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:20.978385 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:20.978385 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:20.979695 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:20.979695 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:20.979695 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:20.979695 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:20.979695 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:20.979695 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:20.979695 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:20.979695 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:20.979695 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:20.979695 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:20.979695 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:30:20.979695 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:20.980418 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-c8de1527-1ede-4883-beba-2bcf5712c557 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:30:20.980418 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:20.980418 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:20.980418 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:20.980418 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:20.980418 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:20.980418 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:20.980418 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:20.980418 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:20.980418 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:20.980418 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:20.980418 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:20.980418 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:30:20.980418 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:30:20.980418 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:30:20.982090 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:30:20.982090 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:20.982090 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:20.982090 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:20.982090 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:20.982090 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:20.982090 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:20.982090 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:20.982090 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:20.982090 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:20.982090 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:20.982090 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:20.982090 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:20.982090 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:20.982090 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:20.982090 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:20.982090 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:20.984406 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:30:20.984406 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:30:20.984406 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:30:20.984406 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:30:20.984406 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:30:20.984406 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:30:20.984406 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:30:20.984406 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:20.985401 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 424/849] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 30 14:30:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:20.985541 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 425/849] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:30:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:20.985669 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f45509-1086-479a-9006-9c0654c41e4a None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:20.985820 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8de1527-1ede-4883-beba-2bcf5712c557 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:21.094937 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8de1527-1ede-4883-beba-2bcf5712c557 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:21.097012 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f45509-1086-479a-9006-9c0654c41e4a None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:21.106874 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:21.133231 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048796d8-a52f-4add-8620-22be87a851e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:21.134125 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048796d8-a52f-4add-8620-22be87a851e8 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:21.134407 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048796d8-a52f-4add-8620-22be87a851e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:21.134721 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048796d8-a52f-4add-8620-22be87a851e8 None admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:21.135510 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048796d8-a52f-4add-8620-22be87a851e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:21.135814 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048796d8-a52f-4add-8620-22be87a851e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:21.137447 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048796d8-a52f-4add-8620-22be87a851e8 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:21.138331 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20177d2c-0884-454a-a06c-98283cf531cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:21.139040 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20177d2c-0884-454a-a06c-98283cf531cd None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:21.139417 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20177d2c-0884-454a-a06c-98283cf531cd None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:21.139775 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20177d2c-0884-454a-a06c-98283cf531cd None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:21.140448 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20177d2c-0884-454a-a06c-98283cf531cd None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:21.141789 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20177d2c-0884-454a-a06c-98283cf531cd None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:21.151042 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 426/850] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 30 14:30:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:21.152111 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048796d8-a52f-4add-8620-22be87a851e8 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:21.156683 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-20177d2c-0884-454a-a06c-98283cf531cd None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 14:30:21.156683 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:21.156683 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:30:21.156683 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:30:21.156683 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:30:21.156683 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:30:21.156683 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:30:21.156683 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:30:21.156683 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 14:30:21.156683 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:21.156683 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:30:21.156683 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:21.156683 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:21.156683 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:21.156683 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:21.156683 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:21.156683 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:21.157790 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:21.157790 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:21.157790 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:21.157790 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:21.157790 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:21.157790 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:21.157790 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:30:21.157790 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:30:21.157790 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:30:21.157790 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:30:21.157790 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:21.157790 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:21.157790 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:21.157790 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:21.157790 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:21.157790 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:21.158642 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:21.158642 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:21.158642 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:21.158642 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:21.158642 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:21.158642 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:21.158642 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:21.158642 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:21.158642 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:21.158642 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:21.158642 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:30:21.158642 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:30:21.158642 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 14:30:21.158642 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:21.163020 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 425/851] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 30 14:30:20 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:21.163787 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20177d2c-0884-454a-a06c-98283cf531cd None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:21.185161 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:21.269428 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048796d8-a52f-4add-8620-22be87a851e8 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:21.282854 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20177d2c-0884-454a-a06c-98283cf531cd None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:21.306049 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0643d861-7129-4c1d-b217-7a1bfb724837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:21.306656 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643d861-7129-4c1d-b217-7a1bfb724837 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:21.307045 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643d861-7129-4c1d-b217-7a1bfb724837 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:21.307540 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643d861-7129-4c1d-b217-7a1bfb724837 None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:21.308448 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0643d861-7129-4c1d-b217-7a1bfb724837 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:21.308779 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0643d861-7129-4c1d-b217-7a1bfb724837 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:21.310374 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0643d861-7129-4c1d-b217-7a1bfb724837 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:21.319141 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 427/852] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:30:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:21.319967 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0643d861-7129-4c1d-b217-7a1bfb724837 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:21.327124 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ae45ab-8c77-4750-963c-0d0cbabd20e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:21.327834 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae45ab-8c77-4750-963c-0d0cbabd20e2 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:21.328119 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae45ab-8c77-4750-963c-0d0cbabd20e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:21.328393 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae45ab-8c77-4750-963c-0d0cbabd20e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:21.329175 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ae45ab-8c77-4750-963c-0d0cbabd20e2 None admin] RBAC: Adding query filter params (role.id=1316db16eac54629a283545f1601a236, scope.project.id=01cf71ca5abf4576b89d10c85af6d2de, user.id=f60bee471e954932a928ab2cf7d66c82) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:21.329485 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ae45ab-8c77-4750-963c-0d0cbabd20e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:21.330958 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ae45ab-8c77-4750-963c-0d0cbabd20e2 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:21.351667 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 426/853] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:30:21 2022] GET /identity/v3/role_assignments?user.id=f60bee471e954932a928ab2cf7d66c82&scope.project.id=01cf71ca5abf4576b89d10c85af6d2de&role.id=1316db16eac54629a283545f1601a236 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:21.352570 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ae45ab-8c77-4750-963c-0d0cbabd20e2 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:21.408176 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:21.433339 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:21.447856 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0643d861-7129-4c1d-b217-7a1bfb724837 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:21.484900 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ae45ab-8c77-4750-963c-0d0cbabd20e2 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:21.488895 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef84596-3762-473b-8c5a-04bf87ae843b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:21.489945 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef84596-3762-473b-8c5a-04bf87ae843b None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:21.489945 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef84596-3762-473b-8c5a-04bf87ae843b None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:21.490133 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef84596-3762-473b-8c5a-04bf87ae843b None admin] PATH_INFO: `/v3/services` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:21.490943 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef84596-3762-473b-8c5a-04bf87ae843b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:21.490943 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef84596-3762-473b-8c5a-04bf87ae843b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:21.492744 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef84596-3762-473b-8c5a-04bf87ae843b None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:21.498805 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 428/854] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 30 14:30:21 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:21.500090 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db684caf-99e5-4353-8671-12801d97f3ef None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:21.500355 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db684caf-99e5-4353-8671-12801d97f3ef None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:21.500933 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db684caf-99e5-4353-8671-12801d97f3ef None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:21.501790 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 429/855] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:21.505048 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db684caf-99e5-4353-8671-12801d97f3ef None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:21.529615 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42dad61-9c05-4152-b8af-6752b8a363e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:21.530260 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42dad61-9c05-4152-b8af-6752b8a363e7 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:21.530544 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42dad61-9c05-4152-b8af-6752b8a363e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:21.530814 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42dad61-9c05-4152-b8af-6752b8a363e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:21.544645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42dad61-9c05-4152-b8af-6752b8a363e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:21.546247 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42dad61-9c05-4152-b8af-6752b8a363e7 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:21.568821 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-d42dad61-9c05-4152-b8af-6752b8a363e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:21.568821 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:21.568821 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:21.568821 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:21.568821 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:21.568821 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:21.568821 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:21.568821 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:21.568821 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:21.568821 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:21.568821 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:21.568821 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:21.568821 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:30:21.568821 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:30:21.568821 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:30:21.568821 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:30:21.569974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:21.569974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:21.569974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:21.569974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:21.569974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:21.569974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:21.569974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:21.569974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:21.569974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:21.569974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:21.569974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:21.569974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:21.569974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:21.569974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:21.569974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:21.569974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:21.569974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:30:21.571048 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:30:21.571048 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:21.571048 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:21.571048 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 427/856] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:21.572226 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57580089-d1f2-47d1-94ce-8e21f378f5bf None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:21.573051 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57580089-d1f2-47d1-94ce-8e21f378f5bf None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:21.573330 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57580089-d1f2-47d1-94ce-8e21f378f5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:21.600078 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-57580089-d1f2-47d1-94ce-8e21f378f5bf None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:21.601283 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-57580089-d1f2-47d1-94ce-8e21f378f5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:21.627120 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db684caf-99e5-4353-8671-12801d97f3ef None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:21.666296 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e2790f-f2de-4e3b-8302-3bacf86638a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:21.667096 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2790f-f2de-4e3b-8302-3bacf86638a9 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:21.667398 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2790f-f2de-4e3b-8302-3bacf86638a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:21.667672 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2790f-f2de-4e3b-8302-3bacf86638a9 None admin] PATH_INFO: `/v3/services` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:21.668338 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e2790f-f2de-4e3b-8302-3bacf86638a9 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:21.668670 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e2790f-f2de-4e3b-8302-3bacf86638a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:21.670048 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e2790f-f2de-4e3b-8302-3bacf86638a9 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:21.674535 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 430/857] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 30 14:30:21 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:30:21.675535 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e2790f-f2de-4e3b-8302-3bacf86638a9 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:21.683430 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 428/858] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:21 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:30:21.705476 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:21.803652 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e2790f-f2de-4e3b-8302-3bacf86638a9 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:21.842369 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cc27a4-f4ce-42b2-8583-5b49b337b5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:21.842976 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cc27a4-f4ce-42b2-8583-5b49b337b5a6 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:21.843362 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cc27a4-f4ce-42b2-8583-5b49b337b5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:21.843646 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cc27a4-f4ce-42b2-8583-5b49b337b5a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:21.844288 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cc27a4-f4ce-42b2-8583-5b49b337b5a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:21.844583 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cc27a4-f4ce-42b2-8583-5b49b337b5a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:21.845943 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cc27a4-f4ce-42b2-8583-5b49b337b5a6 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:21.860083 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 431/859] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:30:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:21.865618 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57580089-d1f2-47d1-94ce-8e21f378f5bf None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:21.899738 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:21.970666 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cc27a4-f4ce-42b2-8583-5b49b337b5a6 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:21.982890 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57580089-d1f2-47d1-94ce-8e21f378f5bf None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:22.021723 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25211068-de78-4664-bfc4-cc335ca4be1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:22.022861 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25211068-de78-4664-bfc4-cc335ca4be1a None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:22.023372 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25211068-de78-4664-bfc4-cc335ca4be1a None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:22.023851 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25211068-de78-4664-bfc4-cc335ca4be1a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:22.032538 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25211068-de78-4664-bfc4-cc335ca4be1a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:22.034443 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25211068-de78-4664-bfc4-cc335ca4be1a None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:22.048950 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-25211068-de78-4664-bfc4-cc335ca4be1a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 14:30:22.048950 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:22.048950 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:22.048950 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:22.048950 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:22.048950 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:22.048950 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:22.048950 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:22.048950 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:22.048950 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:22.048950 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:22.048950 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:22.048950 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:30:22.048950 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:30:22.048950 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:30:22.048950 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:30:22.049958 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:22.049958 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:22.049958 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:30:22.049958 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:30:22.049958 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:30:22.049958 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:30:22.049958 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:22.049958 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:22.049958 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:22.049958 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:22.049958 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:22.049958 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:22.049958 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:22.049958 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:22.049958 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:22.049958 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:22.049958 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:22.049958 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:22.050910 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:22.050910 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:22.050910 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:30:22.050910 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:30:22.050910 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:30:22.050910 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:30:22.050910 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:30:22.050910 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:30:22.050910 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 14:30:22.050910 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:22.053496 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 429/860] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 30 14:30:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:22.059638 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25211068-de78-4664-bfc4-cc335ca4be1a None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:22.061198 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:22.085280 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cc27a4-f4ce-42b2-8583-5b49b337b5a6 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:22.127984 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eaa1ee-c3b5-4d6d-bfb7-112b20bfed71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:22.128630 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eaa1ee-c3b5-4d6d-bfb7-112b20bfed71 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:22.128904 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eaa1ee-c3b5-4d6d-bfb7-112b20bfed71 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:22.129169 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eaa1ee-c3b5-4d6d-bfb7-112b20bfed71 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:22.129759 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eaa1ee-c3b5-4d6d-bfb7-112b20bfed71 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:22.131295 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eaa1ee-c3b5-4d6d-bfb7-112b20bfed71 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:22.142231 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-c6eaa1ee-c3b5-4d6d-bfb7-112b20bfed71 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 14:30:22.142231 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:22.142231 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:30:22.142231 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:30:22.142231 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:30:22.142231 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:30:22.142231 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:30:22.142231 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:30:22.142231 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 14:30:22.142231 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:22.142231 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:30:22.142231 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:22.142231 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:22.142231 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:22.142231 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:22.142231 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:22.143678 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:22.143678 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:22.143678 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:22.143678 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:22.143678 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:22.143678 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:22.143678 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:22.143678 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:30:22.143678 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:30:22.143678 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:30:22.143678 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:30:22.143678 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:22.143678 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:22.143678 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:22.143678 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:22.143678 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:22.143678 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:22.144576 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:22.144576 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:22.144576 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:22.144576 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:22.144576 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:22.144576 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:22.144576 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:22.144576 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:22.144576 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:22.144576 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:22.144576 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:30:22.144576 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:30:22.144576 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 14:30:22.144576 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:22.144576 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 432/861] 10.222.0.30 () {56 vars in 1132 bytes} [Tue Aug 30 14:30:21 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:30:22.149241 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eaa1ee-c3b5-4d6d-bfb7-112b20bfed71 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:22.278033 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eaa1ee-c3b5-4d6d-bfb7-112b20bfed71 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:22.286236 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:22.310974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25211068-de78-4664-bfc4-cc335ca4be1a None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:22.318763 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832acb0c-8f1a-4d62-9f6c-ec8e779f89b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:22.319413 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832acb0c-8f1a-4d62-9f6c-ec8e779f89b3 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:22.319690 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832acb0c-8f1a-4d62-9f6c-ec8e779f89b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:22.319959 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832acb0c-8f1a-4d62-9f6c-ec8e779f89b3 None admin] PATH_INFO: `/v3/services` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:22.320586 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832acb0c-8f1a-4d62-9f6c-ec8e779f89b3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:22.320885 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832acb0c-8f1a-4d62-9f6c-ec8e779f89b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:22.322550 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832acb0c-8f1a-4d62-9f6c-ec8e779f89b3 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:22.332041 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 433/862] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Aug 30 14:30:22 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:30:22.343912 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832acb0c-8f1a-4d62-9f6c-ec8e779f89b3 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:22.352240 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1e7b5b-8a8a-4385-ace3-3cf914cde0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:22.353070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e7b5b-8a8a-4385-ace3-3cf914cde0bc None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:22.354103 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e7b5b-8a8a-4385-ace3-3cf914cde0bc None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:22.354682 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e7b5b-8a8a-4385-ace3-3cf914cde0bc None admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:22.355670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1e7b5b-8a8a-4385-ace3-3cf914cde0bc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:22.356199 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1e7b5b-8a8a-4385-ace3-3cf914cde0bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:22.357852 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1e7b5b-8a8a-4385-ace3-3cf914cde0bc None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:22.370885 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:22.381900 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 430/863] 10.222.0.30 () {56 vars in 1135 bytes} [Tue Aug 30 14:30:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:30:22.383104 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1e7b5b-8a8a-4385-ace3-3cf914cde0bc None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:22.453494 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832acb0c-8f1a-4d62-9f6c-ec8e779f89b3 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:22.488283 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1c741e-8fb3-451c-8877-dc9de58454ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:22.488977 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c741e-8fb3-451c-8877-dc9de58454ae None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:22.489228 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c741e-8fb3-451c-8877-dc9de58454ae None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:22.489498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c741e-8fb3-451c-8877-dc9de58454ae None admin] PATH_INFO: `/v3/services` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:22.490169 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1c741e-8fb3-451c-8877-dc9de58454ae None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:22.490470 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1c741e-8fb3-451c-8877-dc9de58454ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:22.491942 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1c741e-8fb3-451c-8877-dc9de58454ae None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:22.503659 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 434/864] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Aug 30 14:30:22 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:22.509703 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1e7b5b-8a8a-4385-ace3-3cf914cde0bc None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:22.556047 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c54c4-ba58-4131-be93-d66b166a6b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:22.557681 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c54c4-ba58-4131-be93-d66b166a6b44 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:22.558177 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c54c4-ba58-4131-be93-d66b166a6b44 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:22.558680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c54c4-ba58-4131-be93-d66b166a6b44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:22.563575 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:22.635943 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93c54c4-ba58-4131-be93-d66b166a6b44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:22.637409 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93c54c4-ba58-4131-be93-d66b166a6b44 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:22.742915 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 431/865] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:30:22 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:30:22.748854 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1c741e-8fb3-451c-8877-dc9de58454ae None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:22.827846 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:22.861999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1c741e-8fb3-451c-8877-dc9de58454ae None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:22.899551 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f58e5d-15c8-4c46-b03a-2995a4e07d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:22.900193 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f58e5d-15c8-4c46-b03a-2995a4e07d69 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:22.901126 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f58e5d-15c8-4c46-b03a-2995a4e07d69 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:22.901467 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f58e5d-15c8-4c46-b03a-2995a4e07d69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:22.912626 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f58e5d-15c8-4c46-b03a-2995a4e07d69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:22.914172 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f58e5d-15c8-4c46-b03a-2995a4e07d69 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:22.929259 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-07f58e5d-15c8-4c46-b03a-2995a4e07d69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:22.929259 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:22.929259 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:22.929259 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:22.929259 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:22.929259 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:22.929259 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:22.929259 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:22.929259 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:22.929259 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:22.929259 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:22.929259 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:22.929259 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:30:22.929259 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:30:22.929259 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:30:22.929259 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:30:22.930236 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:22.930236 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:22.930236 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:22.930236 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:22.930236 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:22.930236 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:22.930236 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:22.930236 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:22.930236 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:22.930236 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:22.930236 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:22.930236 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:22.930236 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:22.930236 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:22.930236 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:22.930236 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:22.930236 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:30:22.931587 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:30:22.931587 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:22.931587 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:22.931587 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 435/866] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:22.936708 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c54c4-ba58-4131-be93-d66b166a6b44 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:22.997322 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:23.047820 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c54c4-ba58-4131-be93-d66b166a6b44 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:23.083718 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7986e84-2015-40f6-b681-8a7a979b13ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:23.084327 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7986e84-2015-40f6-b681-8a7a979b13ef None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:23.084677 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7986e84-2015-40f6-b681-8a7a979b13ef None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:23.084958 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7986e84-2015-40f6-b681-8a7a979b13ef None admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:23.085664 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7986e84-2015-40f6-b681-8a7a979b13ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:23.085970 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7986e84-2015-40f6-b681-8a7a979b13ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:23.087440 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7986e84-2015-40f6-b681-8a7a979b13ef None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:23.100461 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 432/867] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:30:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:23.106230 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f58e5d-15c8-4c46-b03a-2995a4e07d69 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:23.215316 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f58e5d-15c8-4c46-b03a-2995a4e07d69 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:23.251611 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:23.257860 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed547574-892b-4885-b14b-2092f35be77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:23.258794 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed547574-892b-4885-b14b-2092f35be77b None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:23.259304 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed547574-892b-4885-b14b-2092f35be77b None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:23.259832 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed547574-892b-4885-b14b-2092f35be77b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:23.266891 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed547574-892b-4885-b14b-2092f35be77b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:23.268659 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed547574-892b-4885-b14b-2092f35be77b None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:23.280604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-ed547574-892b-4885-b14b-2092f35be77b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:30:23.280604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:23.280604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:23.280604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:23.280604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:23.280604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:23.280604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:23.280604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:23.280604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:23.280604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:23.280604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:23.280604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:23.280604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:30:23.280604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:30:23.280604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:30:23.281526 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:30:23.281526 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:23.281526 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:23.281526 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:23.281526 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:23.281526 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:23.281526 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:23.281526 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:23.281526 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:23.281526 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:23.281526 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:23.281526 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:23.281526 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:23.281526 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:23.281526 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:23.281526 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:23.281526 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:23.282429 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:30:23.282429 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:30:23.282429 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:30:23.282429 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:30:23.282429 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:30:23.282429 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:30:23.282429 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:30:23.282429 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:23.284816 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 436/868] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:30:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:23.290899 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7986e84-2015-40f6-b681-8a7a979b13ef None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:23.400341 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7986e84-2015-40f6-b681-8a7a979b13ef None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:23.404664 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:23.437837 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43e07b6-6ec7-47be-b1cd-7d8d517865a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:23.438805 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43e07b6-6ec7-47be-b1cd-7d8d517865a9 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:23.439349 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43e07b6-6ec7-47be-b1cd-7d8d517865a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:23.439825 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43e07b6-6ec7-47be-b1cd-7d8d517865a9 None admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:23.440727 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43e07b6-6ec7-47be-b1cd-7d8d517865a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:23.441228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43e07b6-6ec7-47be-b1cd-7d8d517865a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:23.442913 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43e07b6-6ec7-47be-b1cd-7d8d517865a9 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:23.446487 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33999553-9f6c-45ff-ae01-a300d53271c4 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:23.446771 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33999553-9f6c-45ff-ae01-a300d53271c4 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:23.447045 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33999553-9f6c-45ff-ae01-a300d53271c4 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:23.449191 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 437/869] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:30:23.456716 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95317b3b-9b36-4af0-86db-e667bf1a64de None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:23.458479 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95317b3b-9b36-4af0-86db-e667bf1a64de None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:23.458479 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95317b3b-9b36-4af0-86db-e667bf1a64de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:23.465989 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 433/870] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 30 14:30:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:23.471510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43e07b6-6ec7-47be-b1cd-7d8d517865a9 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:23.482675 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-95317b3b-9b36-4af0-86db-e667bf1a64de None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:23.484296 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-95317b3b-9b36-4af0-86db-e667bf1a64de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:23.557551 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 438/871] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:23 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:30:23.606202 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43e07b6-6ec7-47be-b1cd-7d8d517865a9 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:23.645957 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c8b78-a72a-41bf-8077-246f6d05c39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:23.646947 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c8b78-a72a-41bf-8077-246f6d05c39d None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:23.647432 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c8b78-a72a-41bf-8077-246f6d05c39d None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:23.647882 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c8b78-a72a-41bf-8077-246f6d05c39d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:23.648781 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1c8b78-a72a-41bf-8077-246f6d05c39d None admin] RBAC: Adding query filter params (role.id=3cda0615d87d4af0860a2a5e9b4e2c05, scope.project.id=01cf71ca5abf4576b89d10c85af6d2de, user.id=a18b13e3708144e1b5e8df80ec3a4e83) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:23.649265 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1c8b78-a72a-41bf-8077-246f6d05c39d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:23.651274 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1c8b78-a72a-41bf-8077-246f6d05c39d None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:23.658024 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 434/872] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:30:23 2022] GET /identity/v3/role_assignments?user.id=a18b13e3708144e1b5e8df80ec3a4e83&scope.project.id=01cf71ca5abf4576b89d10c85af6d2de&role.id=3cda0615d87d4af0860a2a5e9b4e2c05 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:30:23.716494 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dd9dcf-9ff1-4bf2-914a-9d3645a5104b None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:23.716769 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dd9dcf-9ff1-4bf2-914a-9d3645a5104b None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:23.717038 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dd9dcf-9ff1-4bf2-914a-9d3645a5104b None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:23.718722 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 439/873] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:23.724304 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c26076-54c7-4aea-b550-76bab8135381 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:23.724587 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c26076-54c7-4aea-b550-76bab8135381 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:23.724861 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c26076-54c7-4aea-b550-76bab8135381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:23.746113 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-b6c26076-54c7-4aea-b550-76bab8135381 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:23.747368 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-b6c26076-54c7-4aea-b550-76bab8135381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:23.768413 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:23.833665 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 435/874] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:23 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:30:23.853420 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dd9dcf-9ff1-4bf2-914a-9d3645a5104b None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:23.968932 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dd9dcf-9ff1-4bf2-914a-9d3645a5104b None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:23.986781 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:24.011022 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d490e0c2-331c-4a7b-a8cc-e6a6bb621cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:24.011687 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490e0c2-331c-4a7b-a8cc-e6a6bb621cc3 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:24.011959 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490e0c2-331c-4a7b-a8cc-e6a6bb621cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:24.012228 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490e0c2-331c-4a7b-a8cc-e6a6bb621cc3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:24.025701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d490e0c2-331c-4a7b-a8cc-e6a6bb621cc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:24.032761 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-d490e0c2-331c-4a7b-a8cc-e6a6bb621cc3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 14:30:24.032761 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:24.032761 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:24.032761 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:24.032761 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:24.032761 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:24.032761 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:24.032761 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:24.032761 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:24.032761 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:24.032761 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:24.032761 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:24.032761 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:24.032761 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:24.032761 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:24.032761 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:24.033997 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:24.033997 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:24.033997 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:24.033997 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:24.033997 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:24.033997 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:24.033997 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:24.033997 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:24.033997 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:24.033997 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:24.033997 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:24.033997 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:24.033997 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:24.033997 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:24.033997 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:24.033997 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:24.034864 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:24.034864 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:24.034864 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:24.034864 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:24.034864 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:24.034864 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:24.034864 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:24.034864 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:24.034864 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:24.034864 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:24.034864 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 14:30:24.034864 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:24.034864 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 440/875] 10.222.0.30 () {56 vars in 1120 bytes} [Tue Aug 30 14:30:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:24.039248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c26076-54c7-4aea-b550-76bab8135381 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:24.117215 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d490e0c2-331c-4a7b-a8cc-e6a6bb621cc3 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:24.148766 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c26076-54c7-4aea-b550-76bab8135381 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:24.185796 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856be42a-32d2-4010-9e44-92ed021ec00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:24.186410 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856be42a-32d2-4010-9e44-92ed021ec00f None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:24.186691 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856be42a-32d2-4010-9e44-92ed021ec00f None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:24.186965 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856be42a-32d2-4010-9e44-92ed021ec00f None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:24.187670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856be42a-32d2-4010-9e44-92ed021ec00f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:24.187972 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856be42a-32d2-4010-9e44-92ed021ec00f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:24.189340 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856be42a-32d2-4010-9e44-92ed021ec00f None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:24.198733 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 436/876] 10.222.0.30 () {56 vars in 1115 bytes} [Tue Aug 30 14:30:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:24.204384 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856be42a-32d2-4010-9e44-92ed021ec00f None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:24.221494 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:24.252381 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d490e0c2-331c-4a7b-a8cc-e6a6bb621cc3 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:24.304404 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d245663-2671-44a1-b81f-7ee06744ab92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:24.305614 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d245663-2671-44a1-b81f-7ee06744ab92 None admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:24.305900 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d245663-2671-44a1-b81f-7ee06744ab92 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:24.306173 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d245663-2671-44a1-b81f-7ee06744ab92 None admin] PATH_INFO: `/v3/services` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:24.307410 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d245663-2671-44a1-b81f-7ee06744ab92 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:24.309598 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d245663-2671-44a1-b81f-7ee06744ab92 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:24.333158 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 441/877] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Aug 30 14:30:24 2022] POST /identity/v3/services => generated 251 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:30:24.334615 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856be42a-32d2-4010-9e44-92ed021ec00f None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:24.389215 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c932839-8b9b-413e-ac7b-a389a1f14ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:24.390049 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c932839-8b9b-413e-ac7b-a389a1f14ba6 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:24.390398 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c932839-8b9b-413e-ac7b-a389a1f14ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:24.390784 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c932839-8b9b-413e-ac7b-a389a1f14ba6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:24.404557 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c932839-8b9b-413e-ac7b-a389a1f14ba6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:24.406193 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c932839-8b9b-413e-ac7b-a389a1f14ba6 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:24.423913 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-2c932839-8b9b-413e-ac7b-a389a1f14ba6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:24.423913 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:24.423913 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:24.423913 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:24.423913 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:24.423913 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:24.423913 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:24.423913 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:24.423913 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:24.423913 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:24.423913 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:24.423913 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:24.423913 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:30:24.423913 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:30:24.423913 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:30:24.423913 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:30:24.425100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:24.425100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:24.425100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:24.425100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:24.425100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:24.425100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:24.425100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:24.425100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:24.425100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:24.425100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:24.425100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:24.425100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:24.425100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:24.425100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:24.425100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:24.425100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:24.425100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:30:24.426023 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:30:24.426023 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:24.426023 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:24.426023 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 437/878] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:24.431557 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d245663-2671-44a1-b81f-7ee06744ab92 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:24.469201 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb5f5a-06e4-4298-9c9e-2f11c34377ed None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:24.469558 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb5f5a-06e4-4298-9c9e-2f11c34377ed None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:24.469758 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb5f5a-06e4-4298-9c9e-2f11c34377ed None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:24.479332 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 438/879] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:24.481504 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:24.482430 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f40653-743c-4fba-9a93-82a0db054c77 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:24.482898 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f40653-743c-4fba-9a93-82a0db054c77 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:24.483343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f40653-743c-4fba-9a93-82a0db054c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:24.505477 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-a2f40653-743c-4fba-9a93-82a0db054c77 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:24.506927 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-a2f40653-743c-4fba-9a93-82a0db054c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:24.554718 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d245663-2671-44a1-b81f-7ee06744ab92 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:24.571373 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:24.589955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 439/880] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:24 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:30:24.594383 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475578da-746a-4a1c-9d81-4c1e7e363913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:24.595724 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475578da-746a-4a1c-9d81-4c1e7e363913 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:24.596219 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475578da-746a-4a1c-9d81-4c1e7e363913 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:24.596681 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475578da-746a-4a1c-9d81-4c1e7e363913 None admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:24.597559 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475578da-746a-4a1c-9d81-4c1e7e363913 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:24.598089 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475578da-746a-4a1c-9d81-4c1e7e363913 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:24.599666 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475578da-746a-4a1c-9d81-4c1e7e363913 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:24.613338 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 442/881] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:30:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:24.619246 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f40653-743c-4fba-9a93-82a0db054c77 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:24.729218 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f40653-743c-4fba-9a93-82a0db054c77 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:24.765364 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afe73c3-d1db-40eb-bb74-8e87a12854bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:24.766320 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe73c3-d1db-40eb-bb74-8e87a12854bd None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:24.766810 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe73c3-d1db-40eb-bb74-8e87a12854bd None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:24.767307 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe73c3-d1db-40eb-bb74-8e87a12854bd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:24.774864 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afe73c3-d1db-40eb-bb74-8e87a12854bd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:24.776977 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afe73c3-d1db-40eb-bb74-8e87a12854bd None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:24.789003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-3afe73c3-d1db-40eb-bb74-8e87a12854bd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 14:30:24.789003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:24.789003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:24.789003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:24.789003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:24.789003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:24.789003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:24.789003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:24.789003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:24.789003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:24.789003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:24.789003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:24.789003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:30:24.789003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:30:24.789003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:30:24.789003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:30:24.789983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:24.789983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:24.789983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:30:24.789983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:30:24.789983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:30:24.789983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:30:24.789983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:24.789983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:24.789983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:24.789983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:24.789983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:24.789983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:24.789983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:24.789983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:24.789983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:24.789983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:24.789983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:24.789983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:24.790927 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:24.790927 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:24.790927 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:30:24.790927 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:30:24.790927 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:30:24.790927 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:30:24.790927 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:30:24.790927 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:30:24.790927 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 14:30:24.790927 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:24.794332 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 440/882] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Aug 30 14:30:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:24.799911 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475578da-746a-4a1c-9d81-4c1e7e363913 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:24.864369 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afe73c3-d1db-40eb-bb74-8e87a12854bd None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:24.918268 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:24.922146 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:24.931429 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475578da-746a-4a1c-9d81-4c1e7e363913 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:24.968799 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f9f93-d087-4cd3-8db7-e5b9d5178270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:24.969776 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f9f93-d087-4cd3-8db7-e5b9d5178270 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:24.970324 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f9f93-d087-4cd3-8db7-e5b9d5178270 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:24.970819 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f9f93-d087-4cd3-8db7-e5b9d5178270 None admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:24.971884 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5f9f93-d087-4cd3-8db7-e5b9d5178270 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:24.972407 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5f9f93-d087-4cd3-8db7-e5b9d5178270 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:24.974027 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5f9f93-d087-4cd3-8db7-e5b9d5178270 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:24.983729 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afe73c3-d1db-40eb-bb74-8e87a12854bd None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:24.993274 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 443/883] 10.222.0.30 () {56 vars in 1149 bytes} [Tue Aug 30 14:30:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:24.999082 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f9f93-d087-4cd3-8db7-e5b9d5178270 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:25.018960 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a32ba2-6752-43ac-b524-262c21299a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:25.019813 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a32ba2-6752-43ac-b524-262c21299a3a None admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:25.020237 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a32ba2-6752-43ac-b524-262c21299a3a None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:25.020646 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a32ba2-6752-43ac-b524-262c21299a3a None admin] PATH_INFO: `/v3/services` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:25.021459 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a32ba2-6752-43ac-b524-262c21299a3a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:25.022944 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a32ba2-6752-43ac-b524-262c21299a3a None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:25.059340 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 441/884] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Aug 30 14:30:24 2022] POST /identity/v3/services => generated 269 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:30:25.116563 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f9f93-d087-4cd3-8db7-e5b9d5178270 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:25.163480 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce72313-fb2d-43af-97bd-52c72f193e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:25.164145 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce72313-fb2d-43af-97bd-52c72f193e3c None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:25.164428 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce72313-fb2d-43af-97bd-52c72f193e3c None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:25.164702 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce72313-fb2d-43af-97bd-52c72f193e3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:25.183354 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:25.251488 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce72313-fb2d-43af-97bd-52c72f193e3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:25.253192 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce72313-fb2d-43af-97bd-52c72f193e3c None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:25.365234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 444/885] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:30:24 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:30:25.374426 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a32ba2-6752-43ac-b524-262c21299a3a None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:25.483893 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a32ba2-6752-43ac-b524-262c21299a3a None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:25.503683 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:25.524720 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f86e29f-2be9-4d3f-af2c-12b4c9f20490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:25.525442 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86e29f-2be9-4d3f-af2c-12b4c9f20490 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:25.525742 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86e29f-2be9-4d3f-af2c-12b4c9f20490 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:25.526018 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86e29f-2be9-4d3f-af2c-12b4c9f20490 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:25.538067 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f86e29f-2be9-4d3f-af2c-12b4c9f20490 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:25.539721 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f86e29f-2be9-4d3f-af2c-12b4c9f20490 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:25.554637 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-4f86e29f-2be9-4d3f-af2c-12b4c9f20490 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:25.554637 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:25.554637 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:25.554637 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:25.554637 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:25.554637 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:25.554637 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:25.554637 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:25.554637 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:25.554637 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:25.554637 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:25.554637 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:25.554637 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:30:25.554637 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:30:25.554637 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:30:25.554637 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:30:25.555633 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:25.555633 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:25.555633 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:25.555633 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:25.555633 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:25.555633 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:25.555633 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:25.555633 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:25.555633 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:25.555633 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:25.555633 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:25.555633 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:25.555633 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:25.555633 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:25.555633 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:25.555633 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:25.555633 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:30:25.556592 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:30:25.556592 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:25.556592 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:25.556592 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 442/886] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:25.565885 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce72313-fb2d-43af-97bd-52c72f193e3c None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:25.622788 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:25.679645 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce72313-fb2d-43af-97bd-52c72f193e3c None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:25.725920 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5aae5d-79c5-4fe3-bb0f-7dad0aca7c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:25.725920 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5aae5d-79c5-4fe3-bb0f-7dad0aca7c7d None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:25.725920 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5aae5d-79c5-4fe3-bb0f-7dad0aca7c7d None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:25.725920 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5aae5d-79c5-4fe3-bb0f-7dad0aca7c7d None admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:25.726335 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5aae5d-79c5-4fe3-bb0f-7dad0aca7c7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:25.726335 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5aae5d-79c5-4fe3-bb0f-7dad0aca7c7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:25.731235 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5aae5d-79c5-4fe3-bb0f-7dad0aca7c7d None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:25.740603 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 445/887] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:30:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:25.748332 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f86e29f-2be9-4d3f-af2c-12b4c9f20490 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:25.796752 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c2b4d-3903-48c7-8e03-eb48dde5be65 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:25.797095 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c2b4d-3903-48c7-8e03-eb48dde5be65 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:25.797302 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c2b4d-3903-48c7-8e03-eb48dde5be65 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:25.799954 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 446/888] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:25.804172 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43c07b9-b6c5-47d2-bc9a-c2c097de5a90 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:25.804465 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43c07b9-b6c5-47d2-bc9a-c2c097de5a90 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:25.804740 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43c07b9-b6c5-47d2-bc9a-c2c097de5a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:25.824800 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-e43c07b9-b6c5-47d2-bc9a-c2c097de5a90 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:25.826053 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-e43c07b9-b6c5-47d2-bc9a-c2c097de5a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:25.870942 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f86e29f-2be9-4d3f-af2c-12b4c9f20490 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:25.910549 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 447/889] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:25 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:30:25.920043 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893430c4-88f9-4cd5-94ff-53d89f78b628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:25.920906 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893430c4-88f9-4cd5-94ff-53d89f78b628 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:25.921368 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893430c4-88f9-4cd5-94ff-53d89f78b628 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:25.921807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893430c4-88f9-4cd5-94ff-53d89f78b628 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:25.929384 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893430c4-88f9-4cd5-94ff-53d89f78b628 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:25.931011 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893430c4-88f9-4cd5-94ff-53d89f78b628 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:25.943510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-893430c4-88f9-4cd5-94ff-53d89f78b628 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:30:25.943510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:25.943510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:25.943510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:25.943510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:25.943510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:25.943510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:25.943510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:25.943510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:25.943510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:25.943510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:25.943510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:25.943510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:30:25.943510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:30:25.943510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:30:25.944508 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:30:25.944508 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:25.944508 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:25.944508 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:25.944508 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:25.944508 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:25.944508 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:25.944508 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:25.944508 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:25.944508 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:25.944508 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:25.944508 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:25.944508 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:25.944508 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:25.944508 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:25.944508 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:25.944508 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:25.945403 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:30:25.945403 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:30:25.945403 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:30:25.945403 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:30:25.945403 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:30:25.945403 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:30:25.945403 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:30:25.945403 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:25.947947 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 443/890] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:30:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:25.954232 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43c07b9-b6c5-47d2-bc9a-c2c097de5a90 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:25.999596 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:26.063200 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43c07b9-b6c5-47d2-bc9a-c2c097de5a90 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:26.100424 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae8fdf-5057-43f6-ba71-6ce9ba7458a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:26.101436 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae8fdf-5057-43f6-ba71-6ce9ba7458a7 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:26.101919 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae8fdf-5057-43f6-ba71-6ce9ba7458a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:26.102412 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae8fdf-5057-43f6-ba71-6ce9ba7458a7 None admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:26.103340 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efae8fdf-5057-43f6-ba71-6ce9ba7458a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:26.103892 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efae8fdf-5057-43f6-ba71-6ce9ba7458a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:26.105660 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efae8fdf-5057-43f6-ba71-6ce9ba7458a7 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:26.117963 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 448/891] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 30 14:30:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:26.124219 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893430c4-88f9-4cd5-94ff-53d89f78b628 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:26.163634 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:26.206914 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae8fdf-5057-43f6-ba71-6ce9ba7458a7 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:26.246437 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893430c4-88f9-4cd5-94ff-53d89f78b628 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:26.289193 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b9a5d2-a1c4-4b1a-b760-212c8e81551d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:26.289982 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b9a5d2-a1c4-4b1a-b760-212c8e81551d None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:26.290326 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b9a5d2-a1c4-4b1a-b760-212c8e81551d None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:26.290743 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b9a5d2-a1c4-4b1a-b760-212c8e81551d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:26.292022 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b9a5d2-a1c4-4b1a-b760-212c8e81551d None admin] RBAC: Adding query filter params (role.id=2345dd1fbf154d6888e8113c51914966, scope.project.id=01cf71ca5abf4576b89d10c85af6d2de, user.id=f60bee471e954932a928ab2cf7d66c82) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:26.292437 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b9a5d2-a1c4-4b1a-b760-212c8e81551d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:26.294492 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b9a5d2-a1c4-4b1a-b760-212c8e81551d None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:26.307241 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 444/892] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:30:26 2022] GET /identity/v3/role_assignments?user.id=f60bee471e954932a928ab2cf7d66c82&scope.project.id=01cf71ca5abf4576b89d10c85af6d2de&role.id=2345dd1fbf154d6888e8113c51914966 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:26.331715 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:26.339328 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae8fdf-5057-43f6-ba71-6ce9ba7458a7 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:26.387239 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0216e8d7-2e59-4424-9ab9-08601f28624f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:26.388613 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0216e8d7-2e59-4424-9ab9-08601f28624f None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:26.389224 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0216e8d7-2e59-4424-9ab9-08601f28624f None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:26.389866 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0216e8d7-2e59-4424-9ab9-08601f28624f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:26.390944 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0216e8d7-2e59-4424-9ab9-08601f28624f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:26.392486 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0216e8d7-2e59-4424-9ab9-08601f28624f None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:26.403461 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-0216e8d7-2e59-4424-9ab9-08601f28624f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 14:30:26.403461 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:26.403461 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:30:26.403461 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:30:26.403461 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:30:26.403461 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:30:26.403461 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:30:26.403461 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:30:26.403461 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 14:30:26.403461 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:26.403461 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:30:26.403461 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:26.403461 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:26.403461 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:26.403461 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:26.403461 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:26.403461 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:26.404740 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:26.404740 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:26.404740 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:26.404740 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:26.404740 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:26.404740 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:26.404740 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:30:26.404740 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:30:26.404740 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:30:26.404740 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:30:26.404740 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:26.404740 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:26.404740 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:26.404740 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:26.404740 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:26.404740 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:26.405808 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:26.405808 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:26.405808 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:26.405808 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:26.405808 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:26.405808 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:26.405808 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:26.405808 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:26.405808 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:26.405808 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:26.405808 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:30:26.405808 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:30:26.405808 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 14:30:26.405808 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:26.408552 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 449/893] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Aug 30 14:30:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:26.411883 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f405b00-bc22-4855-9b3e-fa4acc170e8f None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:26.412144 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f405b00-bc22-4855-9b3e-fa4acc170e8f None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:26.412424 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f405b00-bc22-4855-9b3e-fa4acc170e8f None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:26.414599 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 445/894] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:26.415042 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0216e8d7-2e59-4424-9ab9-08601f28624f None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:26.419977 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48d0d3-bdfa-4976-bd6a-84a66e571b18 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:26.421015 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48d0d3-bdfa-4976-bd6a-84a66e571b18 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:26.421015 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48d0d3-bdfa-4976-bd6a-84a66e571b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:26.441438 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-aa48d0d3-bdfa-4976-bd6a-84a66e571b18 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:26.442759 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-aa48d0d3-bdfa-4976-bd6a-84a66e571b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:26.532675 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 446/895] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:26 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:30:26.534615 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0216e8d7-2e59-4424-9ab9-08601f28624f None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:26.574953 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e5ed5-9ea0-4937-b72b-bdb97e979f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:26.576118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e5ed5-9ea0-4937-b72b-bdb97e979f08 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:26.576627 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e5ed5-9ea0-4937-b72b-bdb97e979f08 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:26.577118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e5ed5-9ea0-4937-b72b-bdb97e979f08 None admin] PATH_INFO: `/v3/services` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:26.577960 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e5ed5-9ea0-4937-b72b-bdb97e979f08 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:26.578503 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e5ed5-9ea0-4937-b72b-bdb97e979f08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:26.580089 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e5ed5-9ea0-4937-b72b-bdb97e979f08 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:26.585538 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 450/896] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Aug 30 14:30:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:30:26.593539 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa48d0d3-bdfa-4976-bd6a-84a66e571b18 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:26.615784 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:26.707250 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa48d0d3-bdfa-4976-bd6a-84a66e571b18 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:26.746207 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28836944-6d67-4baa-8e16-783c1aa15d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:26.746832 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28836944-6d67-4baa-8e16-783c1aa15d83 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:26.747121 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28836944-6d67-4baa-8e16-783c1aa15d83 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:26.747421 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28836944-6d67-4baa-8e16-783c1aa15d83 None admin] PATH_INFO: `/v3/services` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:26.748064 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28836944-6d67-4baa-8e16-783c1aa15d83 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:26.748370 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28836944-6d67-4baa-8e16-783c1aa15d83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:26.749746 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28836944-6d67-4baa-8e16-783c1aa15d83 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:26.754750 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 447/897] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Aug 30 14:30:26 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:26.827358 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e5ed5-9ea0-4937-b72b-bdb97e979f08 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:26.859631 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:26.942537 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e5ed5-9ea0-4937-b72b-bdb97e979f08 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:26.980202 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2721412-ebda-4aac-94bc-363390886623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:26.980769 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2721412-ebda-4aac-94bc-363390886623 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:26.981051 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2721412-ebda-4aac-94bc-363390886623 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:26.981324 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2721412-ebda-4aac-94bc-363390886623 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:26.981914 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2721412-ebda-4aac-94bc-363390886623 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:26.983313 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2721412-ebda-4aac-94bc-363390886623 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:26.992433 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-c2721412-ebda-4aac-94bc-363390886623 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 14:30:26.992433 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:26.992433 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:30:26.992433 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:30:26.992433 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:30:26.992433 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:30:26.992433 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:30:26.992433 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:30:26.992433 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 14:30:26.992433 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:26.992433 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:30:26.992433 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:26.992433 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:26.992433 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:26.992433 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:26.992433 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:26.992433 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:26.993424 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:26.993424 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:26.993424 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:26.993424 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:26.993424 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:26.993424 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:26.993424 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:30:26.993424 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:30:26.993424 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:30:26.993424 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:30:26.993424 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:26.993424 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:26.993424 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:26.993424 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:26.993424 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:26.993424 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:26.994346 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:26.994346 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:26.994346 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:26.994346 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:26.994346 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:26.994346 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:26.994346 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:26.994346 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:26.994346 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:26.994346 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:26.994346 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:30:26.994346 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:30:26.994346 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 14:30:26.994346 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:26.994346 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 451/898] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 30 14:30:26 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:27.001510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28836944-6d67-4baa-8e16-783c1aa15d83 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:27.095590 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:27.114331 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28836944-6d67-4baa-8e16-783c1aa15d83 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:27.140598 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f75a5-440a-44de-8339-d52c6efa422b None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:27.141024 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f75a5-440a-44de-8339-d52c6efa422b None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:27.141394 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f75a5-440a-44de-8339-d52c6efa422b None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:27.143062 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 452/899] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:27.147709 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8fe1a5-e4a6-44cb-a7dd-70d9b9eab7f2 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:27.148085 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8fe1a5-e4a6-44cb-a7dd-70d9b9eab7f2 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:27.148455 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8fe1a5-e4a6-44cb-a7dd-70d9b9eab7f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:27.152276 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc399d9-36e2-4083-8db3-2babfa4a2f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:27.152881 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc399d9-36e2-4083-8db3-2babfa4a2f0f None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:27.153161 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc399d9-36e2-4083-8db3-2babfa4a2f0f None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:27.153432 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc399d9-36e2-4083-8db3-2babfa4a2f0f None admin] PATH_INFO: `/v3/services` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:27.154059 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc399d9-36e2-4083-8db3-2babfa4a2f0f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:27.154361 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc399d9-36e2-4083-8db3-2babfa4a2f0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:27.155740 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc399d9-36e2-4083-8db3-2babfa4a2f0f None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:27.160826 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 448/900] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 30 14:30:27 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:27.167047 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc399d9-36e2-4083-8db3-2babfa4a2f0f None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:27.172629 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-0c8fe1a5-e4a6-44cb-a7dd-70d9b9eab7f2 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:27.173944 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-0c8fe1a5-e4a6-44cb-a7dd-70d9b9eab7f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:27.283331 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 453/901] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:27 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:30:27.301350 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc399d9-36e2-4083-8db3-2babfa4a2f0f None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:27.342058 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a697ba-ad90-41de-9539-b2ae17411d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:27.342913 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a697ba-ad90-41de-9539-b2ae17411d2d None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:27.343414 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a697ba-ad90-41de-9539-b2ae17411d2d None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:27.343839 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a697ba-ad90-41de-9539-b2ae17411d2d None admin] PATH_INFO: `/v3/services` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:27.344693 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a697ba-ad90-41de-9539-b2ae17411d2d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:27.345143 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a697ba-ad90-41de-9539-b2ae17411d2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:27.346654 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a697ba-ad90-41de-9539-b2ae17411d2d None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:27.352340 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 449/902] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 30 14:30:27 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:30:27.358763 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8fe1a5-e4a6-44cb-a7dd-70d9b9eab7f2 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:27.423592 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:27.468451 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8fe1a5-e4a6-44cb-a7dd-70d9b9eab7f2 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:27.506891 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d7a8a1-f308-4f80-b6a5-221274f69577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:27.507538 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d7a8a1-f308-4f80-b6a5-221274f69577 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:27.507822 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d7a8a1-f308-4f80-b6a5-221274f69577 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:27.508094 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d7a8a1-f308-4f80-b6a5-221274f69577 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:27.508801 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d7a8a1-f308-4f80-b6a5-221274f69577 None admin] RBAC: Adding query filter params (interface=public, service_id=aefac0d16a9747df858dc38775139e0a, region_id=RegionOne) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:27.509109 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d7a8a1-f308-4f80-b6a5-221274f69577 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:27.511173 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d7a8a1-f308-4f80-b6a5-221274f69577 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:27.516983 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 454/903] 10.222.0.30 () {56 vars in 1251 bytes} [Tue Aug 30 14:30:27 2022] GET /identity/v3/endpoints?service_id=aefac0d16a9747df858dc38775139e0a&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:27.522135 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a697ba-ad90-41de-9539-b2ae17411d2d None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:27.535675 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:27.577031 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d7a8a1-f308-4f80-b6a5-221274f69577 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:27.648437 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a697ba-ad90-41de-9539-b2ae17411d2d None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:27.685509 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be4e14-37b0-411b-9864-84b676cc36cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:27.686148 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be4e14-37b0-411b-9864-84b676cc36cd None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:27.686284 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:27.686453 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be4e14-37b0-411b-9864-84b676cc36cd None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:27.686691 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be4e14-37b0-411b-9864-84b676cc36cd None admin] PATH_INFO: `/v3/services` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:27.687402 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74be4e14-37b0-411b-9864-84b676cc36cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:27.688836 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74be4e14-37b0-411b-9864-84b676cc36cd None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:27.698397 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d7a8a1-f308-4f80-b6a5-221274f69577 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:27.703661 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 450/904] 10.222.0.30 () {56 vars in 1087 bytes} [Tue Aug 30 14:30:27 2022] GET /identity/v3/services => generated 1268 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:30:27.742005 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06a8abd-7f23-40b2-8708-de5a33c44334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:27.743509 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06a8abd-7f23-40b2-8708-de5a33c44334 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:27.744056 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06a8abd-7f23-40b2-8708-de5a33c44334 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:27.745110 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06a8abd-7f23-40b2-8708-de5a33c44334 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:27.745942 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06a8abd-7f23-40b2-8708-de5a33c44334 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:27.748610 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06a8abd-7f23-40b2-8708-de5a33c44334 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:27.759348 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-b06a8abd-7f23-40b2-8708-de5a33c44334 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 14:30:27.759348 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:27.759348 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:30:27.759348 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:30:27.759348 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:30:27.759348 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:30:27.759348 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:30:27.759348 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:30:27.759348 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 14:30:27.759348 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:27.759348 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:30:27.759348 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:27.759348 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:27.759348 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:27.759348 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:27.759348 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:27.760431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:27.760431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:27.760431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:27.760431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:27.760431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:27.760431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:27.760431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:27.760431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:30:27.760431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:30:27.760431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:30:27.760431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:30:27.760431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:27.760431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:27.760431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:27.760431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:27.760431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:27.760431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:27.761332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:27.761332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:27.761332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:27.761332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:27.761332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:27.761332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:27.761332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:27.761332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:27.761332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:27.761332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:27.761332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:30:27.761332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:30:27.761332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 14:30:27.761332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:27.764263 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 455/905] 10.222.0.30 () {56 vars in 1132 bytes} [Tue Aug 30 14:30:27 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:30:27.775308 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be4e14-37b0-411b-9864-84b676cc36cd None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:27.870044 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:27.897578 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be4e14-37b0-411b-9864-84b676cc36cd None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:27.943960 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deef591-f2e9-4698-a3a6-c7526ede30d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:27.944583 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deef591-f2e9-4698-a3a6-c7526ede30d0 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:27.944859 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deef591-f2e9-4698-a3a6-c7526ede30d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:27.945127 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deef591-f2e9-4698-a3a6-c7526ede30d0 None admin] PATH_INFO: `/v3/services` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:27.945762 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deef591-f2e9-4698-a3a6-c7526ede30d0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:27.946059 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deef591-f2e9-4698-a3a6-c7526ede30d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:27.948121 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deef591-f2e9-4698-a3a6-c7526ede30d0 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:27.953981 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 451/906] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Aug 30 14:30:27 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:27.959537 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06a8abd-7f23-40b2-8708-de5a33c44334 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:28.069908 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06a8abd-7f23-40b2-8708-de5a33c44334 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:28.111233 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccefa26-672a-4dd6-8e8f-a3ae17a33ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:28.111233 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccefa26-672a-4dd6-8e8f-a3ae17a33ed1 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:28.111233 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccefa26-672a-4dd6-8e8f-a3ae17a33ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:28.111233 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccefa26-672a-4dd6-8e8f-a3ae17a33ed1 None admin] PATH_INFO: `/v3/services` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:28.111653 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccefa26-672a-4dd6-8e8f-a3ae17a33ed1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:28.111653 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccefa26-672a-4dd6-8e8f-a3ae17a33ed1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:28.111794 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccefa26-672a-4dd6-8e8f-a3ae17a33ed1 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:28.118109 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 456/907] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Aug 30 14:30:27 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:28.124162 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deef591-f2e9-4698-a3a6-c7526ede30d0 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:28.237679 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deef591-f2e9-4698-a3a6-c7526ede30d0 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:28.274778 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf7c0f5-29c9-49eb-bce3-821a18614c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:28.276086 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf7c0f5-29c9-49eb-bce3-821a18614c58 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:28.276589 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf7c0f5-29c9-49eb-bce3-821a18614c58 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:28.276985 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf7c0f5-29c9-49eb-bce3-821a18614c58 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:28.277687 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf7c0f5-29c9-49eb-bce3-821a18614c58 None admin] RBAC: Adding query filter params (interface=public, service_id=ba5cf002b62944d58dd4a5225c37f4a5, region_id=RegionOne) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:28.277991 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf7c0f5-29c9-49eb-bce3-821a18614c58 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:28.279376 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf7c0f5-29c9-49eb-bce3-821a18614c58 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:28.281780 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:28.285983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 452/908] 10.222.0.30 () {56 vars in 1251 bytes} [Tue Aug 30 14:30:28 2022] GET /identity/v3/endpoints?service_id=ba5cf002b62944d58dd4a5225c37f4a5&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:28.292616 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccefa26-672a-4dd6-8e8f-a3ae17a33ed1 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:28.306377 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:28.352731 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd6ffad-07c6-4dc8-9758-266f5b25bfaf None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:28.353044 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd6ffad-07c6-4dc8-9758-266f5b25bfaf None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:28.353267 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd6ffad-07c6-4dc8-9758-266f5b25bfaf None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:28.355453 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 453/909] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:28.360502 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7bf126-dbbe-4bef-a743-adcd10c27d85 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:28.360788 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7bf126-dbbe-4bef-a743-adcd10c27d85 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:28.361057 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7bf126-dbbe-4bef-a743-adcd10c27d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:28.383588 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-ac7bf126-dbbe-4bef-a743-adcd10c27d85 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:28.385812 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-ac7bf126-dbbe-4bef-a743-adcd10c27d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:28.410859 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccefa26-672a-4dd6-8e8f-a3ae17a33ed1 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:28.455678 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d62c1a-48b7-4355-bc85-9b7706d64685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:28.456295 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d62c1a-48b7-4355-bc85-9b7706d64685 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:28.456573 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d62c1a-48b7-4355-bc85-9b7706d64685 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:28.456845 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d62c1a-48b7-4355-bc85-9b7706d64685 None admin] PATH_INFO: `/v3/services` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:28.457504 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d62c1a-48b7-4355-bc85-9b7706d64685 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:28.459395 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d62c1a-48b7-4355-bc85-9b7706d64685 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:28.468186 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 457/910] 10.222.0.30 () {56 vars in 1087 bytes} [Tue Aug 30 14:30:28 2022] GET /identity/v3/services => generated 1268 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:30:28.483403 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 454/911] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:28 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:30:28.754230 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d62c1a-48b7-4355-bc85-9b7706d64685 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:28.788362 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f3341-d0c4-4701-9e1b-5561b70cba35 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:28.788685 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f3341-d0c4-4701-9e1b-5561b70cba35 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:28.788913 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f3341-d0c4-4701-9e1b-5561b70cba35 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:28.792145 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 455/912] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:28.796362 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc866dd3-340d-41c2-8529-464128945734 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:28.796642 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc866dd3-340d-41c2-8529-464128945734 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:28.796974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc866dd3-340d-41c2-8529-464128945734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:28.816209 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-cc866dd3-340d-41c2-8529-464128945734 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:28.817602 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-cc866dd3-340d-41c2-8529-464128945734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:28.847260 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:28.883180 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d62c1a-48b7-4355-bc85-9b7706d64685 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:28.911435 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 456/913] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:28 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:30:28.926673 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acb1406-e0d8-469b-a240-703d21c47ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:28.927722 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb1406-e0d8-469b-a240-703d21c47ce6 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:28.928200 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb1406-e0d8-469b-a240-703d21c47ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:28.928665 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb1406-e0d8-469b-a240-703d21c47ce6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:28.939977 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acb1406-e0d8-469b-a240-703d21c47ce6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:28.945426 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-1acb1406-e0d8-469b-a240-703d21c47ce6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 14:30:28.945426 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:28.945426 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:28.945426 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:28.945426 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:28.945426 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:28.945426 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:28.945426 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:28.945426 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:28.945426 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:28.945426 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:28.945426 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:28.945426 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:28.945426 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:28.945426 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:28.945426 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:28.946362 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:28.946362 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:28.946362 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:28.946362 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:28.946362 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:28.946362 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:28.946362 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:28.946362 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:28.946362 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:28.946362 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:28.946362 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:28.946362 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:28.946362 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:28.946362 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:28.946362 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:28.946362 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:28.947778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:28.947778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:28.947778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:28.947778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:28.947778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:28.947778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:28.947778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:28.947778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:28.947778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:28.947778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:28.947778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 14:30:28.947778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:28.950328 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 458/914] 10.222.0.30 () {56 vars in 1120 bytes} [Tue Aug 30 14:30:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:28.956425 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc866dd3-340d-41c2-8529-464128945734 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:29.024597 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:29.061195 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc866dd3-340d-41c2-8529-464128945734 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:29.095940 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cac2e98-ddcd-4617-937a-0a6b4dc23639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:29.096911 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac2e98-ddcd-4617-937a-0a6b4dc23639 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:29.097435 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac2e98-ddcd-4617-937a-0a6b4dc23639 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:29.097927 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac2e98-ddcd-4617-937a-0a6b4dc23639 None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:29.098812 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cac2e98-ddcd-4617-937a-0a6b4dc23639 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:29.099343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cac2e98-ddcd-4617-937a-0a6b4dc23639 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:29.100913 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cac2e98-ddcd-4617-937a-0a6b4dc23639 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:29.110646 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 457/915] 10.222.0.30 () {56 vars in 1115 bytes} [Tue Aug 30 14:30:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:30:29.116606 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acb1406-e0d8-469b-a240-703d21c47ce6 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:29.187028 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cac2e98-ddcd-4617-937a-0a6b4dc23639 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:29.239614 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acb1406-e0d8-469b-a240-703d21c47ce6 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:29.284381 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa34d6e-9f09-4a5b-bc1c-edadf96c4f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:29.284988 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa34d6e-9f09-4a5b-bc1c-edadf96c4f34 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:29.285267 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa34d6e-9f09-4a5b-bc1c-edadf96c4f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:29.285536 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa34d6e-9f09-4a5b-bc1c-edadf96c4f34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:29.287827 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:29.299920 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa34d6e-9f09-4a5b-bc1c-edadf96c4f34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:29.301537 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa34d6e-9f09-4a5b-bc1c-edadf96c4f34 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:29.318331 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cac2e98-ddcd-4617-937a-0a6b4dc23639 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:29.320374 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-8aa34d6e-9f09-4a5b-bc1c-edadf96c4f34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:29.320374 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:29.320374 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:29.320374 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:29.320374 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:29.320374 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:29.320374 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:29.320374 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:29.320374 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:29.320374 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:29.320374 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:29.320374 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:29.320374 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:30:29.320374 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:30:29.320374 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:30:29.320374 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:30:29.321310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:29.321310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:29.321310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:29.321310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:29.321310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:29.321310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:29.321310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:29.321310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:29.321310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:29.321310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:29.321310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:29.321310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:29.321310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:29.321310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:29.321310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:29.321310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:29.321310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:30:29.322248 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:30:29.322248 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:29.322248 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:29.323283 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 459/916] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:29.328972 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa34d6e-9f09-4a5b-bc1c-edadf96c4f34 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:29.355364 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:29.379283 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188f0db8-7120-4197-adb8-176cbc374822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:29.379283 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f0db8-7120-4197-adb8-176cbc374822 None admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:29.379283 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f0db8-7120-4197-adb8-176cbc374822 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:29.379283 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f0db8-7120-4197-adb8-176cbc374822 None admin] PATH_INFO: `/v3/services` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:29.379701 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188f0db8-7120-4197-adb8-176cbc374822 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:29.380555 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188f0db8-7120-4197-adb8-176cbc374822 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:29.398768 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 458/917] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Aug 30 14:30:29 2022] POST /identity/v3/services => generated 241 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:30:29.452728 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa34d6e-9f09-4a5b-bc1c-edadf96c4f34 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:29.497432 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a24e9-9e2f-42bb-bb61-4d92da9f54e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:29.498047 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a24e9-9e2f-42bb-bb61-4d92da9f54e1 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:29.498322 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a24e9-9e2f-42bb-bb61-4d92da9f54e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:29.498591 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a24e9-9e2f-42bb-bb61-4d92da9f54e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:29.499252 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9a24e9-9e2f-42bb-bb61-4d92da9f54e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:29.499553 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9a24e9-9e2f-42bb-bb61-4d92da9f54e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:29.500926 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9a24e9-9e2f-42bb-bb61-4d92da9f54e1 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:29.519850 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 460/918] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:30:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:29.521241 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188f0db8-7120-4197-adb8-176cbc374822 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:29.641991 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188f0db8-7120-4197-adb8-176cbc374822 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:29.679991 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d660f-d2b7-478b-b5e9-278c7affd320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:29.680625 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d660f-d2b7-478b-b5e9-278c7affd320 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:29.680942 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d660f-d2b7-478b-b5e9-278c7affd320 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:29.681219 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d660f-d2b7-478b-b5e9-278c7affd320 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:29.691326 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198d660f-d2b7-478b-b5e9-278c7affd320 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:29.692803 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198d660f-d2b7-478b-b5e9-278c7affd320 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:29.704739 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-198d660f-d2b7-478b-b5e9-278c7affd320 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 14:30:29.704739 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:29.704739 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:29.704739 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:29.704739 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:29.704739 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:29.704739 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:29.704739 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:29.704739 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:29.704739 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:29.704739 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:29.704739 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:29.704739 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:30:29.704739 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:30:29.704739 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:30:29.704739 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:30:29.705710 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:29.705710 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:29.705710 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:30:29.705710 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:30:29.705710 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:30:29.705710 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:30:29.705710 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:29.705710 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:29.705710 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:29.705710 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:29.705710 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:29.705710 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:29.705710 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:29.705710 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:29.705710 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:29.705710 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:29.705710 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:29.705710 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:29.707283 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:29.707283 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:29.707283 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:30:29.707283 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:30:29.707283 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:30:29.707283 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:30:29.707283 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:30:29.707283 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:30:29.707283 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 14:30:29.707283 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:29.707283 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 459/919] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Aug 30 14:30:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:29.711823 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a24e9-9e2f-42bb-bb61-4d92da9f54e1 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:29.767656 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:29.782792 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d6e62c-8140-4f8e-ab4e-24b7c4319f84 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:29.783067 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d6e62c-8140-4f8e-ab4e-24b7c4319f84 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:29.783364 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d6e62c-8140-4f8e-ab4e-24b7c4319f84 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:29.785541 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 460/920] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:29.793518 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da33a7c9-cd17-46d9-ac61-4a8de88963c7 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:29.794012 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da33a7c9-cd17-46d9-ac61-4a8de88963c7 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:29.794475 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da33a7c9-cd17-46d9-ac61-4a8de88963c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:29.814045 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-da33a7c9-cd17-46d9-ac61-4a8de88963c7 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:29.815526 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-da33a7c9-cd17-46d9-ac61-4a8de88963c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:29.828611 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a24e9-9e2f-42bb-bb61-4d92da9f54e1 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:29.864374 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:29.867196 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cd7d0d-f0df-4250-b742-724eb8001e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:29.868364 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd7d0d-f0df-4250-b742-724eb8001e0f None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:29.868846 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd7d0d-f0df-4250-b742-724eb8001e0f None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:29.869876 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd7d0d-f0df-4250-b742-724eb8001e0f None admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:29.872276 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cd7d0d-f0df-4250-b742-724eb8001e0f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:29.873139 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cd7d0d-f0df-4250-b742-724eb8001e0f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:29.875973 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cd7d0d-f0df-4250-b742-724eb8001e0f None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:29.901171 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 461/921] 10.222.0.30 () {56 vars in 1149 bytes} [Tue Aug 30 14:30:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:29.906545 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cd7d0d-f0df-4250-b742-724eb8001e0f None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:29.915304 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 461/922] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:30:30.011814 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:30.014658 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cd7d0d-f0df-4250-b742-724eb8001e0f None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:30.052672 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f999b27-8d4f-4762-9f87-30f74c0eefd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:30.053306 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f999b27-8d4f-4762-9f87-30f74c0eefd0 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:30.053589 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f999b27-8d4f-4762-9f87-30f74c0eefd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:30.053858 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f999b27-8d4f-4762-9f87-30f74c0eefd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:30.069184 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f999b27-8d4f-4762-9f87-30f74c0eefd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:30.070641 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f999b27-8d4f-4762-9f87-30f74c0eefd0 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:30.085885 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-3f999b27-8d4f-4762-9f87-30f74c0eefd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:30.085885 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:30.085885 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:30.085885 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:30.085885 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:30.085885 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:30.085885 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:30.085885 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:30.085885 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:30.085885 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:30.085885 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:30.085885 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:30.085885 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:30:30.085885 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:30:30.085885 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:30:30.085885 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:30:30.086842 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:30.086842 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:30.086842 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:30.086842 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:30.086842 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:30.086842 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:30.086842 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:30.086842 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:30.086842 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:30.086842 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:30.086842 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:30.086842 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:30.086842 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:30.086842 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:30.086842 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:30.086842 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:30.086842 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:30:30.088957 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:30:30.088957 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:30.088957 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:30.088957 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 462/923] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:30:30.096303 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da33a7c9-cd17-46d9-ac61-4a8de88963c7 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:30.200735 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f999b27-8d4f-4762-9f87-30f74c0eefd0 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:30.212398 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da33a7c9-cd17-46d9-ac61-4a8de88963c7 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:30.255181 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef23f6-5ac8-4cc0-a80e-a47ceda4277a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:30.256192 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef23f6-5ac8-4cc0-a80e-a47ceda4277a None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:30.256677 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef23f6-5ac8-4cc0-a80e-a47ceda4277a None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:30.257140 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef23f6-5ac8-4cc0-a80e-a47ceda4277a None admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:30.257986 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ef23f6-5ac8-4cc0-a80e-a47ceda4277a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:30.258478 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ef23f6-5ac8-4cc0-a80e-a47ceda4277a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:30.260664 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ef23f6-5ac8-4cc0-a80e-a47ceda4277a None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:30.273039 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 462/924] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:30:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:30.281150 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef23f6-5ac8-4cc0-a80e-a47ceda4277a None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:30.316764 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:30.332351 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f999b27-8d4f-4762-9f87-30f74c0eefd0 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:30.368462 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fdd060-5301-44c5-9237-34d1a347c5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:30.369990 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fdd060-5301-44c5-9237-34d1a347c5ee None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:30.370474 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fdd060-5301-44c5-9237-34d1a347c5ee None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:30.370971 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fdd060-5301-44c5-9237-34d1a347c5ee None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:30.371810 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fdd060-5301-44c5-9237-34d1a347c5ee None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:30.373381 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fdd060-5301-44c5-9237-34d1a347c5ee None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:30.382664 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-78fdd060-5301-44c5-9237-34d1a347c5ee None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 14:30:30.382664 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:30.382664 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:30:30.382664 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:30:30.382664 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:30:30.382664 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:30:30.382664 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:30:30.382664 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:30:30.382664 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 14:30:30.382664 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:30.382664 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:30:30.382664 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:30.382664 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:30.382664 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:30.382664 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:30.382664 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:30.382664 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:30.383686 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:30.383686 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:30.383686 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:30.383686 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:30.383686 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:30.383686 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:30.383686 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:30:30.383686 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:30:30.383686 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:30:30.383686 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:30:30.383686 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:30.383686 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:30.383686 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:30.383686 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:30.383686 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:30.383686 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:30.384510 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:30.384510 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:30.384510 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:30.384510 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:30.384510 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:30.384510 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:30.384510 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:30.384510 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:30.384510 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:30.384510 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:30.384510 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:30:30.384510 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:30:30.384510 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 14:30:30.384510 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:30.387526 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 463/925] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 30 14:30:30 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:30:30.393646 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fdd060-5301-44c5-9237-34d1a347c5ee None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:30.402469 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef23f6-5ac8-4cc0-a80e-a47ceda4277a None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:30.415615 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:30.453703 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f529410-2aef-4a98-a98d-c7c73935f387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:30.455066 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f529410-2aef-4a98-a98d-c7c73935f387 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:30.457333 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f529410-2aef-4a98-a98d-c7c73935f387 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:30.457766 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f529410-2aef-4a98-a98d-c7c73935f387 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:30.467005 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f529410-2aef-4a98-a98d-c7c73935f387 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:30.469105 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f529410-2aef-4a98-a98d-c7c73935f387 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:30.484493 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-8f529410-2aef-4a98-a98d-c7c73935f387 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:30:30.484493 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:30.484493 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:30.484493 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:30.484493 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:30.484493 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:30.484493 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:30.484493 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:30.484493 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:30.484493 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:30.484493 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:30.484493 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:30.484493 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:30:30.484493 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:30:30.484493 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:30:30.485589 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:30:30.485589 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:30.485589 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:30.485589 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:30.485589 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:30.485589 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:30.485589 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:30.485589 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:30.485589 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:30.485589 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:30.485589 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:30.485589 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:30.485589 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:30.485589 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:30.485589 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:30.485589 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:30.485589 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:30.486678 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:30:30.486678 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:30:30.486678 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:30:30.486678 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:30:30.486678 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:30:30.486678 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:30:30.486678 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:30:30.486678 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:30.493387 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 463/926] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:30:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:30:30.495338 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f529410-2aef-4a98-a98d-c7c73935f387 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:30.518670 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fdd060-5301-44c5-9237-34d1a347c5ee None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:30.556247 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1127b197-7cfe-4c28-b41b-8db4b79973f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:30.558246 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127b197-7cfe-4c28-b41b-8db4b79973f6 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:30.558739 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127b197-7cfe-4c28-b41b-8db4b79973f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:30.560268 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127b197-7cfe-4c28-b41b-8db4b79973f6 None admin] PATH_INFO: `/v3/services` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:30.561116 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1127b197-7cfe-4c28-b41b-8db4b79973f6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:30.561634 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1127b197-7cfe-4c28-b41b-8db4b79973f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:30.563253 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1127b197-7cfe-4c28-b41b-8db4b79973f6 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:30.573131 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 464/927] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 30 14:30:30 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:30:30.573131 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d799484-ce4a-4902-b053-56dcaaf98567 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:30.573131 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d799484-ce4a-4902-b053-56dcaaf98567 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:30.573131 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d799484-ce4a-4902-b053-56dcaaf98567 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:30.574301 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 465/928] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:30.574859 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d799484-ce4a-4902-b053-56dcaaf98567 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:30.601988 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:30.613408 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f529410-2aef-4a98-a98d-c7c73935f387 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:30.648052 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3412b87-5317-4f89-ab74-7833ed4fabe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:30.648732 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3412b87-5317-4f89-ab74-7833ed4fabe7 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:30.648915 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3412b87-5317-4f89-ab74-7833ed4fabe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:30.649173 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3412b87-5317-4f89-ab74-7833ed4fabe7 None admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:30.649858 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3412b87-5317-4f89-ab74-7833ed4fabe7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:30.650190 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3412b87-5317-4f89-ab74-7833ed4fabe7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:30.651635 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3412b87-5317-4f89-ab74-7833ed4fabe7 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:30.665224 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 464/929] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 30 14:30:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:30.666472 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1c91c-cef8-4b0c-9bbc-9959a2165c36 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:30.666756 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1c91c-cef8-4b0c-9bbc-9959a2165c36 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:30.667022 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1c91c-cef8-4b0c-9bbc-9959a2165c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:30.683831 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:30.688389 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-30d1c91c-cef8-4b0c-9bbc-9959a2165c36 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:30.689570 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-30d1c91c-cef8-4b0c-9bbc-9959a2165c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:30.705616 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d799484-ce4a-4902-b053-56dcaaf98567 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:30.741423 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba723c7-1c6f-433d-a93a-f7974a0fe6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:30.742269 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba723c7-1c6f-433d-a93a-f7974a0fe6de None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:30.742745 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba723c7-1c6f-433d-a93a-f7974a0fe6de None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:30.743199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba723c7-1c6f-433d-a93a-f7974a0fe6de None admin] PATH_INFO: `/v3/services` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:30.744723 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba723c7-1c6f-433d-a93a-f7974a0fe6de None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:30.745418 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba723c7-1c6f-433d-a93a-f7974a0fe6de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:30.746963 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba723c7-1c6f-433d-a93a-f7974a0fe6de None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:30.752083 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 466/930] 10.222.0.30 () {56 vars in 1114 bytes} [Tue Aug 30 14:30:30 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:30.752833 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba723c7-1c6f-433d-a93a-f7974a0fe6de None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:30.766269 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 465/931] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:30:30.767327 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1c91c-cef8-4b0c-9bbc-9959a2165c36 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:30.863949 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:30.882826 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba723c7-1c6f-433d-a93a-f7974a0fe6de None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:30.888326 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1c91c-cef8-4b0c-9bbc-9959a2165c36 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:30.923244 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02471f50-3bd3-4028-90e4-1aeb76a86e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:30.923915 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02471f50-3bd3-4028-90e4-1aeb76a86e38 None admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:30.924190 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02471f50-3bd3-4028-90e4-1aeb76a86e38 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:30.924462 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02471f50-3bd3-4028-90e4-1aeb76a86e38 None admin] PATH_INFO: `/v3/projects/01cf71ca5abf4576b89d10c85af6d2de/users/f60bee471e954932a928ab2cf7d66c82/roles/2345dd1fbf154d6888e8113c51914966` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:30.926620 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02471f50-3bd3-4028-90e4-1aeb76a86e38 None admin] RBAC: Authorizing `identity:create_grant(project_id=01cf71ca5abf4576b89d10c85af6d2de, user_id=f60bee471e954932a928ab2cf7d66c82, role_id=2345dd1fbf154d6888e8113c51914966)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:30.927664 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865180ac-be5f-43f0-a50c-e80cba63e53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:30.928199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02471f50-3bd3-4028-90e4-1aeb76a86e38 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:30.928703 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865180ac-be5f-43f0-a50c-e80cba63e53f None admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:30.929194 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865180ac-be5f-43f0-a50c-e80cba63e53f None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:30.929658 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865180ac-be5f-43f0-a50c-e80cba63e53f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:30.930518 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865180ac-be5f-43f0-a50c-e80cba63e53f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:30.932959 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865180ac-be5f-43f0-a50c-e80cba63e53f None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:30.951248 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 467/932] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 30 14:30:30 2022] PUT /identity/v3/projects/01cf71ca5abf4576b89d10c85af6d2de/users/f60bee471e954932a928ab2cf7d66c82/roles/2345dd1fbf154d6888e8113c51914966 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:30:30.981478 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 466/933] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:30:30 2022] POST /identity/v3/endpoints => generated 346 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:30:31.053268 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02471f50-3bd3-4028-90e4-1aeb76a86e38 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:31.060203 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:31.183015 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02471f50-3bd3-4028-90e4-1aeb76a86e38 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:31.240199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ef5709-bcd6-4440-b95d-daa641ef9ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:31.240199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef5709-bcd6-4440-b95d-daa641ef9ccb None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:31.240199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef5709-bcd6-4440-b95d-daa641ef9ccb None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:31.240199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef5709-bcd6-4440-b95d-daa641ef9ccb None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:31.240996 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ef5709-bcd6-4440-b95d-daa641ef9ccb None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:31.240996 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ef5709-bcd6-4440-b95d-daa641ef9ccb None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:31.256761 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-d7ef5709-bcd6-4440-b95d-daa641ef9ccb None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 14:30:31.256761 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:31.256761 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:30:31.256761 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:30:31.256761 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:30:31.256761 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:30:31.256761 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:30:31.256761 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:30:31.256761 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 14:30:31.256761 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:31.256761 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:30:31.256761 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:31.256761 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:31.256761 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:31.256761 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:31.256761 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:31.257742 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:31.257742 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:31.257742 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:31.257742 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:31.257742 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:31.257742 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:31.257742 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:31.257742 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:30:31.257742 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:30:31.257742 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:30:31.257742 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:30:31.257742 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:31.257742 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:31.257742 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:31.257742 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:31.257742 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:31.257742 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:31.258668 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:31.258668 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:31.258668 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:31.258668 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:31.258668 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:31.258668 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:31.258668 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:31.258668 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:31.258668 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:31.258668 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:31.258668 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:30:31.258668 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:30:31.258668 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 14:30:31.258668 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:31.262799 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 468/934] 10.222.0.30 () {56 vars in 1132 bytes} [Tue Aug 30 14:30:31 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:31.268825 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865180ac-be5f-43f0-a50c-e80cba63e53f None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:31.382674 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865180ac-be5f-43f0-a50c-e80cba63e53f None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:31.419514 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9ac1e-92b3-4acc-ac54-925ed35fecca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:31.420165 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9ac1e-92b3-4acc-ac54-925ed35fecca None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:31.420430 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9ac1e-92b3-4acc-ac54-925ed35fecca None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:31.421425 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9ac1e-92b3-4acc-ac54-925ed35fecca None admin] PATH_INFO: `/v3/services` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:31.421425 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f9ac1e-92b3-4acc-ac54-925ed35fecca None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:31.421596 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f9ac1e-92b3-4acc-ac54-925ed35fecca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:31.422904 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f9ac1e-92b3-4acc-ac54-925ed35fecca None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:31.428333 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 467/935] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Aug 30 14:30:31 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:31.435115 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ef5709-bcd6-4440-b95d-daa641ef9ccb None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:31.488324 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6962937d-3334-4f5f-958f-6ac3eb457eea None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:31.488956 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6962937d-3334-4f5f-958f-6ac3eb457eea None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:31.489462 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6962937d-3334-4f5f-958f-6ac3eb457eea None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:31.491328 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 468/936] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:31.497220 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92f617-5804-4483-8c04-1b3ff75ee22b None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:31.497800 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92f617-5804-4483-8c04-1b3ff75ee22b None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:31.498286 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92f617-5804-4483-8c04-1b3ff75ee22b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:31.510133 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:31.513333 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:31.521476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-4b92f617-5804-4483-8c04-1b3ff75ee22b None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:31.523517 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-4b92f617-5804-4483-8c04-1b3ff75ee22b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:31.555415 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ef5709-bcd6-4440-b95d-daa641ef9ccb None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:31.596859 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e1bbf9-ba5f-49e9-acc0-6880a7125554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:31.597466 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e1bbf9-ba5f-49e9-acc0-6880a7125554 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:31.597741 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e1bbf9-ba5f-49e9-acc0-6880a7125554 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:31.598113 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e1bbf9-ba5f-49e9-acc0-6880a7125554 None admin] PATH_INFO: `/v3/services` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:31.598834 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e1bbf9-ba5f-49e9-acc0-6880a7125554 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:31.599137 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e1bbf9-ba5f-49e9-acc0-6880a7125554 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:31.600531 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e1bbf9-ba5f-49e9-acc0-6880a7125554 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:31.607236 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 469/937] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Aug 30 14:30:31 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:31.612475 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e1bbf9-ba5f-49e9-acc0-6880a7125554 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:31.625779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 469/938] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:30:31.725535 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e1bbf9-ba5f-49e9-acc0-6880a7125554 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:31.753616 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:31.762238 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267fac7d-35c9-463b-9b9e-2f1948e29606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:31.762887 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267fac7d-35c9-463b-9b9e-2f1948e29606 None admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:31.763165 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267fac7d-35c9-463b-9b9e-2f1948e29606 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:31.763456 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267fac7d-35c9-463b-9b9e-2f1948e29606 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:31.764120 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267fac7d-35c9-463b-9b9e-2f1948e29606 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:31.765470 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267fac7d-35c9-463b-9b9e-2f1948e29606 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:31.788620 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 470/939] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:30:31 2022] POST /identity/v3/endpoints => generated 347 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:30:31.920335 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b92f617-5804-4483-8c04-1b3ff75ee22b None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:32.039192 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b92f617-5804-4483-8c04-1b3ff75ee22b None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:32.079030 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:32.112014 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7272d6eb-4d46-44ed-aa7b-7b60933fd485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:32.112671 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272d6eb-4d46-44ed-aa7b-7b60933fd485 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:32.112953 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272d6eb-4d46-44ed-aa7b-7b60933fd485 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:32.113227 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272d6eb-4d46-44ed-aa7b-7b60933fd485 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:32.113816 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7272d6eb-4d46-44ed-aa7b-7b60933fd485 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:32.115179 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7272d6eb-4d46-44ed-aa7b-7b60933fd485 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:32.127314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-7272d6eb-4d46-44ed-aa7b-7b60933fd485 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 14:30:32.127314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:32.127314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:30:32.127314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:30:32.127314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:30:32.127314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:30:32.127314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:30:32.127314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:30:32.127314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 14:30:32.127314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:32.127314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:30:32.127314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:32.127314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:32.127314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:32.127314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:32.127314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:32.127314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:32.128625 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:32.128625 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:32.128625 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:32.128625 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:32.128625 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:32.128625 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:32.128625 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:30:32.128625 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:30:32.128625 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:30:32.128625 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:30:32.128625 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:32.128625 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:32.128625 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:32.128625 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:32.128625 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:32.128625 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:32.129719 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:32.129719 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:32.129719 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:32.129719 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:32.129719 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:32.129719 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:32.129719 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:32.129719 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:32.129719 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:32.129719 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:32.129719 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:30:32.129719 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:30:32.129719 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 14:30:32.129719 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:32.129719 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 470/940] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Aug 30 14:30:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:32.135250 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267fac7d-35c9-463b-9b9e-2f1948e29606 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:32.254885 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267fac7d-35c9-463b-9b9e-2f1948e29606 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:32.290574 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e7141-9f85-4aca-926c-46ae69731f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:32.292144 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e7141-9f85-4aca-926c-46ae69731f4a None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:32.292672 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e7141-9f85-4aca-926c-46ae69731f4a None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:32.293189 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e7141-9f85-4aca-926c-46ae69731f4a None admin] PATH_INFO: `/v3/services` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:32.294252 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3e7141-9f85-4aca-926c-46ae69731f4a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:32.294787 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3e7141-9f85-4aca-926c-46ae69731f4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:32.296459 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3e7141-9f85-4aca-926c-46ae69731f4a None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:32.303589 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 471/941] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Aug 30 14:30:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:30:32.312660 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7272d6eb-4d46-44ed-aa7b-7b60933fd485 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:32.452980 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7272d6eb-4d46-44ed-aa7b-7b60933fd485 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:32.489775 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3807f4c5-9c04-4825-9cd4-2f5021681b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:32.490392 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3807f4c5-9c04-4825-9cd4-2f5021681b7e None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:32.490674 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3807f4c5-9c04-4825-9cd4-2f5021681b7e None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:32.490948 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3807f4c5-9c04-4825-9cd4-2f5021681b7e None admin] PATH_INFO: `/v3/services` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:32.491620 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3807f4c5-9c04-4825-9cd4-2f5021681b7e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:32.491928 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3807f4c5-9c04-4825-9cd4-2f5021681b7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:32.495238 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3807f4c5-9c04-4825-9cd4-2f5021681b7e None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:32.500628 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 471/942] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Aug 30 14:30:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:32.506516 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e7141-9f85-4aca-926c-46ae69731f4a None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:32.572749 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:32.631863 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e7141-9f85-4aca-926c-46ae69731f4a None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:32.672242 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151c2840-d799-4358-a188-64db19ecf533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:32.672869 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151c2840-d799-4358-a188-64db19ecf533 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:32.673152 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151c2840-d799-4358-a188-64db19ecf533 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:32.673429 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151c2840-d799-4358-a188-64db19ecf533 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:32.679230 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151c2840-d799-4358-a188-64db19ecf533 None admin] RBAC: Adding query filter params (interface=public, service_id=a45fcd7e95f74c5ab6be5d0379d50311, region_id=RegionOne) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:32.679230 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151c2840-d799-4358-a188-64db19ecf533 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:32.679230 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151c2840-d799-4358-a188-64db19ecf533 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:32.682723 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 472/943] 10.222.0.30 () {56 vars in 1251 bytes} [Tue Aug 30 14:30:32 2022] GET /identity/v3/endpoints?service_id=a45fcd7e95f74c5ab6be5d0379d50311&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:32.691193 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3807f4c5-9c04-4825-9cd4-2f5021681b7e None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:32.765037 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:32.812948 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3807f4c5-9c04-4825-9cd4-2f5021681b7e None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:32.852930 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848bbffa-d5d3-4c82-8fc9-c334b7a68a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:32.858718 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848bbffa-d5d3-4c82-8fc9-c334b7a68a9b None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:32.858718 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848bbffa-d5d3-4c82-8fc9-c334b7a68a9b None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:32.858718 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848bbffa-d5d3-4c82-8fc9-c334b7a68a9b None admin] PATH_INFO: `/v3/services` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:32.858718 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848bbffa-d5d3-4c82-8fc9-c334b7a68a9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:32.858718 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848bbffa-d5d3-4c82-8fc9-c334b7a68a9b None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:32.864518 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 472/944] 10.222.0.30 () {56 vars in 1087 bytes} [Tue Aug 30 14:30:32 2022] GET /identity/v3/services => generated 1497 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:30:33.107385 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9df548-0826-420b-a183-4971a5bac9c7 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:33.107385 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9df548-0826-420b-a183-4971a5bac9c7 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:33.107385 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9df548-0826-420b-a183-4971a5bac9c7 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:33.107385 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 473/945] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:33.111703 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a8b56d-121f-4574-9b80-12e4477bd2a4 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:33.111986 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a8b56d-121f-4574-9b80-12e4477bd2a4 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:33.112269 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a8b56d-121f-4574-9b80-12e4477bd2a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:33.135128 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-21a8b56d-121f-4574-9b80-12e4477bd2a4 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:33.136388 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-21a8b56d-121f-4574-9b80-12e4477bd2a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:33.217135 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a6e000-a656-4ffd-89c2-59033129edd3 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:33.217415 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a6e000-a656-4ffd-89c2-59033129edd3 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:33.217686 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a6e000-a656-4ffd-89c2-59033129edd3 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:33.219036 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 474/946] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:30:33.221181 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 473/947] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:30:33.230541 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d66bf-f75e-49a4-85be-1452ab02f310 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:33.230812 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d66bf-f75e-49a4-85be-1452ab02f310 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:33.231082 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d66bf-f75e-49a4-85be-1452ab02f310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:33.252415 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-0d6d66bf-f75e-49a4-85be-1452ab02f310 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:33.253588 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-0d6d66bf-f75e-49a4-85be-1452ab02f310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:33.340356 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 475/948] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:30:33.543089 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a8b56d-121f-4574-9b80-12e4477bd2a4 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:33.618484 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:33.654842 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6d66bf-f75e-49a4-85be-1452ab02f310 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:33.680144 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a8b56d-121f-4574-9b80-12e4477bd2a4 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:33.730868 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3340cdfc-1bad-4cd8-9cbd-758259cc8b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:33.731902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3340cdfc-1bad-4cd8-9cbd-758259cc8b96 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:33.732184 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3340cdfc-1bad-4cd8-9cbd-758259cc8b96 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:33.732456 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3340cdfc-1bad-4cd8-9cbd-758259cc8b96 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:33.745729 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3340cdfc-1bad-4cd8-9cbd-758259cc8b96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:33.750599 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-3340cdfc-1bad-4cd8-9cbd-758259cc8b96 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 14:30:33.750599 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:33.750599 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:33.750599 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:33.750599 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:33.750599 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:33.750599 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:33.750599 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:33.750599 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:33.750599 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:33.750599 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:33.750599 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:33.750599 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:33.750599 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:33.750599 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:33.750599 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:33.751576 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:33.751576 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:33.751576 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:33.751576 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:33.751576 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:33.751576 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:33.751576 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:33.751576 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:33.751576 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:33.751576 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:33.751576 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:33.751576 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:33.751576 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:33.751576 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:33.751576 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:33.751576 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:33.752439 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:33.752439 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:33.752439 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:33.752439 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:33.752439 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:33.752439 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:33.752439 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:33.752439 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:33.752439 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:33.752439 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:33.752439 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 14:30:33.752439 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:33.759638 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 474/949] 10.222.0.30 () {56 vars in 1120 bytes} [Tue Aug 30 14:30:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:33.759873 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:33.767520 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3340cdfc-1bad-4cd8-9cbd-758259cc8b96 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:33.849042 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6d66bf-f75e-49a4-85be-1452ab02f310 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:33.899979 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0588cf27-d6ef-41e7-bd7d-669921751489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:33.900876 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588cf27-d6ef-41e7-bd7d-669921751489 None admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:33.901289 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588cf27-d6ef-41e7-bd7d-669921751489 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:33.901558 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588cf27-d6ef-41e7-bd7d-669921751489 None admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:33.902215 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0588cf27-d6ef-41e7-bd7d-669921751489 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:33.903670 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0588cf27-d6ef-41e7-bd7d-669921751489 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:33.906283 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: WARNING keystone.server.flask.common [None req-0588cf27-d6ef-41e7-bd7d-669921751489 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 30 14:30:33.943512 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.sql.core [None req-0588cf27-d6ef-41e7-bd7d-669921751489 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 30 14:30:33.943512 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 30 14:30:33.943512 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [parameters: {'id': '94184688345a429d83b4f95412a0c65f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 30 14:30:33.943512 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81690) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 30 14:30:33.976891 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-0588cf27-d6ef-41e7-bd7d-669921751489 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 94184688345a429d83b4f95412a0c65f.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 94184688345a429d83b4f95412a0c65f. Aug 30 14:30:33.976891 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:33.976891 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 14:30:33.976891 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 14:30:33.976891 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 14:30:33.976891 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 14:30:33.976891 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 14:30:33.976891 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 14:30:33.976891 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 14:30:33.976891 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application conn.query(q) Aug 30 14:30:33.976891 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 14:30:33.976891 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 14:30:33.976891 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 14:30:33.977767 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application result.read() Aug 30 14:30:33.977767 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 14:30:33.977767 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 14:30:33.977767 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 14:30:33.977767 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 14:30:33.977767 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 14:30:33.977767 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 14:30:33.977767 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 14:30:33.977767 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 14:30:33.977767 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 30 14:30:33.977767 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:33.977767 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 30 14:30:33.977767 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:33.977767 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:33.977767 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 30 14:30:33.977767 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 30 14:30:33.977767 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 30 14:30:33.977767 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 30 14:30:33.978792 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 14:30:33.978792 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application next(self.gen) Aug 30 14:30:33.978792 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 30 14:30:33.978792 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application yield resource Aug 30 14:30:33.978792 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 14:30:33.978792 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application next(self.gen) Aug 30 14:30:33.978792 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 30 14:30:33.978792 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application self.session.rollback() Aug 30 14:30:33.978792 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 30 14:30:33.978792 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application self.force_reraise() Aug 30 14:30:33.978792 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 30 14:30:33.978792 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise self.value Aug 30 14:30:33.978792 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 30 14:30:33.978792 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 30 14:30:33.978792 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 30 14:30:33.978792 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application session.commit() Aug 30 14:30:33.978792 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 30 14:30:33.981234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 30 14:30:33.981234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 30 14:30:33.981234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application self._prepare_impl() Aug 30 14:30:33.981234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 30 14:30:33.981234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application self.session.flush() Aug 30 14:30:33.981234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 30 14:30:33.981234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application self._flush(objects) Aug 30 14:30:33.981234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 30 14:30:33.981234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 30 14:30:33.981234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 30 14:30:33.981234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application compat.raise_( Aug 30 14:30:33.981234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 14:30:33.981234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception Aug 30 14:30:33.981234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 30 14:30:33.981234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application flush_context.execute() Aug 30 14:30:33.981234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 30 14:30:33.981234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rec.execute(self) Aug 30 14:30:33.982115 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 30 14:30:33.982115 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 30 14:30:33.982115 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 30 14:30:33.982115 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 30 14:30:33.982115 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 30 14:30:33.982115 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 30 14:30:33.982115 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 30 14:30:33.982115 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 30 14:30:33.982115 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 30 14:30:33.982115 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 30 14:30:33.982115 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 30 14:30:33.982115 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 30 14:30:33.982115 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 30 14:30:33.982115 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 30 14:30:33.982115 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 30 14:30:33.984903 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 30 14:30:33.984903 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 14:30:33.984903 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception Aug 30 14:30:33.984903 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 14:30:33.984903 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 14:30:33.984903 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 14:30:33.984903 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 14:30:33.984903 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 14:30:33.984903 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 14:30:33.984903 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 14:30:33.984903 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application conn.query(q) Aug 30 14:30:33.984903 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 14:30:33.984903 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 14:30:33.984903 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 14:30:33.984903 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application result.read() Aug 30 14:30:33.984903 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 14:30:33.984903 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 14:30:33.986406 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 14:30:33.986406 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 14:30:33.986406 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 14:30:33.986406 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 14:30:33.986406 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 14:30:33.986406 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 14:30:33.986406 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 30 14:30:33.986406 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 30 14:30:33.986406 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [parameters: {'id': '94184688345a429d83b4f95412a0c65f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 30 14:30:33.986406 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 30 14:30:33.986406 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:33.986406 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:30:33.986406 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:33.986406 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:33.986406 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 30 14:30:33.987199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 30 14:30:33.987199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 30 14:30:33.987199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 30 14:30:33.987199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 30 14:30:33.987199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:33.987199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:30:33.987199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:33.987199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:33.987199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:33.987199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:33.987199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:33.987199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:33.987199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:33.987199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:33.987199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:33.987199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:33.987199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:33.988121 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:33.988121 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 30 14:30:33.988121 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 30 14:30:33.988121 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:33.988121 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:33.988121 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 30 14:30:33.988121 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 30 14:30:33.988121 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 94184688345a429d83b4f95412a0c65f. Aug 30 14:30:33.988121 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:33.988121 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 476/950] 10.222.0.30 () {60 vars in 1140 bytes} [Tue Aug 30 14:30:33 2022] POST /identity/v3/projects => generated 262 bytes in 331 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:33.988121 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0588cf27-d6ef-41e7-bd7d-669921751489 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:33.990968 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3340cdfc-1bad-4cd8-9cbd-758259cc8b96 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:34.014342 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:34.071344 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce65716-51ca-4130-905f-5b7dc328c6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:34.072009 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce65716-51ca-4130-905f-5b7dc328c6dd None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:34.072866 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce65716-51ca-4130-905f-5b7dc328c6dd None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:34.073141 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce65716-51ca-4130-905f-5b7dc328c6dd None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:34.073821 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce65716-51ca-4130-905f-5b7dc328c6dd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:34.074121 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce65716-51ca-4130-905f-5b7dc328c6dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:34.076155 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce65716-51ca-4130-905f-5b7dc328c6dd None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:34.094234 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 475/951] 10.222.0.30 () {56 vars in 1115 bytes} [Tue Aug 30 14:30:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:34.096279 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b02178a-08a2-4749-9f6b-179d97e87bd0 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:34.096650 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b02178a-08a2-4749-9f6b-179d97e87bd0 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:34.096999 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b02178a-08a2-4749-9f6b-179d97e87bd0 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:34.101418 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 476/952] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:30:34.102193 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b02178a-08a2-4749-9f6b-179d97e87bd0 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:34.123725 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:34.127888 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0588cf27-d6ef-41e7-bd7d-669921751489 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:34.178768 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e8f971-d72d-4866-bbe0-15ab19f69d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:34.179431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8f971-d72d-4866-bbe0-15ab19f69d3b None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:34.179978 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8f971-d72d-4866-bbe0-15ab19f69d3b None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:34.180251 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8f971-d72d-4866-bbe0-15ab19f69d3b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:34.188696 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e8f971-d72d-4866-bbe0-15ab19f69d3b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:34.190505 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e8f971-d72d-4866-bbe0-15ab19f69d3b None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:34.204146 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-45e8f971-d72d-4866-bbe0-15ab19f69d3b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:30:34.204146 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:34.204146 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:34.204146 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:34.204146 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:34.204146 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:34.204146 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:34.204146 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:34.204146 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:34.204146 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:34.204146 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:34.204146 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:34.204146 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:30:34.204146 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:30:34.204146 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:30:34.205080 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:30:34.205080 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:34.205080 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:34.205080 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:34.205080 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:34.205080 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:34.205080 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:34.205080 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:34.205080 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:34.205080 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:34.205080 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:34.205080 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:34.205080 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:34.205080 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:34.205080 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:34.205080 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:34.205080 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:34.206138 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:30:34.206138 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:30:34.206138 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:30:34.206138 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:30:34.206138 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:30:34.206138 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:30:34.206138 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:30:34.206138 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:34.206138 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 477/953] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:30:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:34.206665 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290e83d-2572-4966-8996-3e75caaf3dd1 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:34.208280 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290e83d-2572-4966-8996-3e75caaf3dd1 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:34.208280 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290e83d-2572-4966-8996-3e75caaf3dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:34.230267 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-b290e83d-2572-4966-8996-3e75caaf3dd1 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:34.232550 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-b290e83d-2572-4966-8996-3e75caaf3dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:34.256807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b02178a-08a2-4749-9f6b-179d97e87bd0 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:34.301026 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a72f98-4121-4f39-a06b-12af797ec25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:34.302053 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a72f98-4121-4f39-a06b-12af797ec25d None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:34.302554 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a72f98-4121-4f39-a06b-12af797ec25d None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:34.303458 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a72f98-4121-4f39-a06b-12af797ec25d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:34.318283 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a72f98-4121-4f39-a06b-12af797ec25d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:34.320126 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a72f98-4121-4f39-a06b-12af797ec25d None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:34.323692 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 478/954] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:30:34.324645 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b290e83d-2572-4966-8996-3e75caaf3dd1 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:34.341519 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-74a72f98-4121-4f39-a06b-12af797ec25d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:34.341519 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:34.341519 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:34.341519 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:34.341519 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:34.341519 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:34.341519 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:34.341519 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:34.341519 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:34.341519 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:34.341519 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:34.341519 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:34.341519 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:30:34.341519 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:30:34.341519 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:30:34.341519 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:30:34.342572 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:34.342572 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:34.342572 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:34.342572 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:34.342572 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:34.342572 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:34.342572 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:34.342572 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:34.342572 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:34.342572 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:34.342572 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:34.342572 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:34.342572 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:34.342572 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:34.342572 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:34.342572 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:34.342572 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:30:34.343537 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:30:34.343537 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:34.343537 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:34.351133 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 477/955] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:34.352880 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a72f98-4121-4f39-a06b-12af797ec25d None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:34.388428 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:34.439897 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:34.456340 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b290e83d-2572-4966-8996-3e75caaf3dd1 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:34.488149 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a72f98-4121-4f39-a06b-12af797ec25d None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:34.499090 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9939926-42cf-4aae-bd77-af1ec9bda9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:34.500021 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9939926-42cf-4aae-bd77-af1ec9bda9ac None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:34.501895 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9939926-42cf-4aae-bd77-af1ec9bda9ac None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:34.502362 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9939926-42cf-4aae-bd77-af1ec9bda9ac None admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:34.503243 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9939926-42cf-4aae-bd77-af1ec9bda9ac None admin] RBAC: Adding query filter params (name=service) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:34.503749 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9939926-42cf-4aae-bd77-af1ec9bda9ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:34.505596 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9939926-42cf-4aae-bd77-af1ec9bda9ac None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:34.533179 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 479/956] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Aug 30 14:30:34 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:34.563540 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c797f8e-0272-4d74-9ae4-85f24afc9c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:34.564792 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c797f8e-0272-4d74-9ae4-85f24afc9c35 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:34.565281 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c797f8e-0272-4d74-9ae4-85f24afc9c35 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:34.565725 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c797f8e-0272-4d74-9ae4-85f24afc9c35 None admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:34.566558 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c797f8e-0272-4d74-9ae4-85f24afc9c35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:34.567046 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c797f8e-0272-4d74-9ae4-85f24afc9c35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:34.568622 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c797f8e-0272-4d74-9ae4-85f24afc9c35 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:34.600067 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 478/957] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:30:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:34.608636 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9939926-42cf-4aae-bd77-af1ec9bda9ac None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:34.638595 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c797f8e-0272-4d74-9ae4-85f24afc9c35 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:34.768653 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9939926-42cf-4aae-bd77-af1ec9bda9ac None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:34.788139 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:34.817341 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c797f8e-0272-4d74-9ae4-85f24afc9c35 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:34.829325 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba06f7-522b-4daf-939b-587578719c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:34.829325 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba06f7-522b-4daf-939b-587578719c95 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:34.829325 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba06f7-522b-4daf-939b-587578719c95 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:34.829672 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba06f7-522b-4daf-939b-587578719c95 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:34.847154 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ba06f7-522b-4daf-939b-587578719c95 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:34.848862 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ba06f7-522b-4daf-939b-587578719c95 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:34.869249 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcc1d21-7f4e-46e1-94e1-50fa5eb1b764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:34.869800 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc1d21-7f4e-46e1-94e1-50fa5eb1b764 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:34.870077 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc1d21-7f4e-46e1-94e1-50fa5eb1b764 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:34.870348 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc1d21-7f4e-46e1-94e1-50fa5eb1b764 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:34.871154 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcc1d21-7f4e-46e1-94e1-50fa5eb1b764 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:34.872624 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcc1d21-7f4e-46e1-94e1-50fa5eb1b764 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:34.873374 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-23ba06f7-522b-4daf-939b-587578719c95 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 14:30:34.873374 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:34.873374 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:34.873374 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:34.873374 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:34.873374 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:34.873374 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:34.873374 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:34.873374 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:34.873374 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:34.873374 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:34.873374 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:34.873374 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:30:34.873374 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:30:34.873374 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:30:34.873374 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:30:34.874404 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:34.874404 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:34.874404 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:30:34.874404 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:30:34.874404 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:30:34.874404 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:30:34.874404 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:34.874404 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:34.874404 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:34.874404 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:34.874404 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:34.874404 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:34.874404 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:34.874404 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:34.874404 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:34.874404 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:34.874404 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:34.874404 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:34.876354 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:34.876354 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:34.876354 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:30:34.876354 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:30:34.876354 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:30:34.876354 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:30:34.876354 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:30:34.876354 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:30:34.876354 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 14:30:34.876354 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:34.876354 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 480/958] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Aug 30 14:30:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:34.881296 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba06f7-522b-4daf-939b-587578719c95 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:34.890873 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-7dcc1d21-7f4e-46e1-94e1-50fa5eb1b764 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 14:30:34.890873 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:34.890873 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:30:34.890873 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:30:34.890873 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:30:34.890873 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:30:34.890873 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:30:34.890873 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:30:34.890873 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 14:30:34.890873 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:34.890873 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:30:34.890873 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:34.890873 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:34.890873 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:34.890873 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:34.890873 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:34.890873 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:34.892219 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:34.892219 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:34.892219 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:34.892219 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:34.892219 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:34.892219 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:34.892219 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:30:34.892219 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:30:34.892219 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:30:34.892219 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:30:34.892219 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:34.892219 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:34.892219 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:34.892219 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:34.892219 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:34.892219 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:34.893075 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:34.893075 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:34.893075 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:34.893075 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:34.893075 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:34.893075 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:34.893075 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:34.893075 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:34.893075 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:34.893075 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:34.893075 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:30:34.893075 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:30:34.893075 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 14:30:34.893075 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:34.893075 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 479/959] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:30:34 2022] GET /identity/v3/services/compute => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:30:34.898507 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcc1d21-7f4e-46e1-94e1-50fa5eb1b764 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:34.907560 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:35.046881 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba06f7-522b-4daf-939b-587578719c95 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:35.074369 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcc1d21-7f4e-46e1-94e1-50fa5eb1b764 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:35.104107 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ada4c5-13e7-4915-a22f-650f2a067877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:35.104744 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ada4c5-13e7-4915-a22f-650f2a067877 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:35.105026 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ada4c5-13e7-4915-a22f-650f2a067877 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:35.105300 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ada4c5-13e7-4915-a22f-650f2a067877 None admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:35.106473 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ada4c5-13e7-4915-a22f-650f2a067877 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:35.106806 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ada4c5-13e7-4915-a22f-650f2a067877 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:35.108249 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ada4c5-13e7-4915-a22f-650f2a067877 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:35.118661 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc396e6-0d30-42c6-81f6-983f9336a5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:35.121065 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc396e6-0d30-42c6-81f6-983f9336a5d4 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:35.121578 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc396e6-0d30-42c6-81f6-983f9336a5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:35.122086 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc396e6-0d30-42c6-81f6-983f9336a5d4 None admin] PATH_INFO: `/v3/services` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:35.122933 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc396e6-0d30-42c6-81f6-983f9336a5d4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:35.123482 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc396e6-0d30-42c6-81f6-983f9336a5d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:35.125096 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc396e6-0d30-42c6-81f6-983f9336a5d4 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:35.134566 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 481/960] 10.222.0.30 () {56 vars in 1149 bytes} [Tue Aug 30 14:30:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:35.134566 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2deff9-370d-40ad-82be-4208673af612 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:35.134566 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2deff9-370d-40ad-82be-4208673af612 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:35.134566 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2deff9-370d-40ad-82be-4208673af612 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:35.134566 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 482/961] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:35.138593 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2deff9-370d-40ad-82be-4208673af612 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:35.142048 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 480/962] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Aug 30 14:30:34 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:35.143522 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd171d9e-73c5-47a2-b99b-cfa92fda52de None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:35.143886 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd171d9e-73c5-47a2-b99b-cfa92fda52de None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:35.144396 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd171d9e-73c5-47a2-b99b-cfa92fda52de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:35.173694 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-dd171d9e-73c5-47a2-b99b-cfa92fda52de None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:35.175063 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-dd171d9e-73c5-47a2-b99b-cfa92fda52de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:35.184366 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:35.238645 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:35.257820 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 481/963] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:30:35.267520 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd171d9e-73c5-47a2-b99b-cfa92fda52de None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:35.292906 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2deff9-370d-40ad-82be-4208673af612 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:35.374294 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcef3b2-0dcc-4539-9f90-ec8bf4e2a039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:35.375305 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcef3b2-0dcc-4539-9f90-ec8bf4e2a039 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:35.375795 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcef3b2-0dcc-4539-9f90-ec8bf4e2a039 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:35.376294 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcef3b2-0dcc-4539-9f90-ec8bf4e2a039 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:35.413678 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd171d9e-73c5-47a2-b99b-cfa92fda52de None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:35.462840 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7cdfdb-16b3-42bd-b659-cf2223bcbcaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:35.463491 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7cdfdb-16b3-42bd-b659-cf2223bcbcaa None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:35.464019 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7cdfdb-16b3-42bd-b659-cf2223bcbcaa None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:35.464619 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7cdfdb-16b3-42bd-b659-cf2223bcbcaa None admin] PATH_INFO: `/v3/services` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:35.465126 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7cdfdb-16b3-42bd-b659-cf2223bcbcaa None admin] RBAC: Adding query filter params (type=compute) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:35.465439 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7cdfdb-16b3-42bd-b659-cf2223bcbcaa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:35.467628 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7cdfdb-16b3-42bd-b659-cf2223bcbcaa None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:35.483242 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 482/964] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Aug 30 14:30:35 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:35.503240 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcef3b2-0dcc-4539-9f90-ec8bf4e2a039 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:35.503240 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcef3b2-0dcc-4539-9f90-ec8bf4e2a039 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:35.545591 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7cdfdb-16b3-42bd-b659-cf2223bcbcaa None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:35.555637 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:35.584213 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:35.661008 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 483/965] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:30:35 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:30:35.661008 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcef3b2-0dcc-4539-9f90-ec8bf4e2a039 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:35.707311 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7cdfdb-16b3-42bd-b659-cf2223bcbcaa None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:35.751580 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fcc7ee-9048-45a8-827c-b8bbf020d25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:35.751580 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fcc7ee-9048-45a8-827c-b8bbf020d25e None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:35.751580 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fcc7ee-9048-45a8-827c-b8bbf020d25e None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:35.751580 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fcc7ee-9048-45a8-827c-b8bbf020d25e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:35.752053 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fcc7ee-9048-45a8-827c-b8bbf020d25e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:35.752053 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fcc7ee-9048-45a8-827c-b8bbf020d25e None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:35.763664 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-d7fcc7ee-9048-45a8-827c-b8bbf020d25e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 14:30:35.763664 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:35.763664 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:30:35.763664 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:30:35.763664 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:30:35.763664 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:30:35.763664 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:30:35.763664 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:30:35.763664 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 14:30:35.763664 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:35.763664 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:30:35.763664 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:35.763664 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:35.763664 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:35.763664 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:35.763664 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:35.763664 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:35.764955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:35.764955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:35.764955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:35.764955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:35.764955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:35.764955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:35.764955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:30:35.764955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:30:35.764955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:30:35.764955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:30:35.764955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:35.764955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:35.764955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:35.764955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:35.764955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:35.764955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:35.765835 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:35.765835 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:35.765835 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:35.765835 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:35.765835 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:35.765835 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:35.765835 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:35.765835 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:35.765835 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:35.765835 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:35.765835 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:30:35.765835 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:30:35.765835 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 14:30:35.765835 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:35.766595 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 483/966] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Aug 30 14:30:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:30:35.772379 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fcc7ee-9048-45a8-827c-b8bbf020d25e None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:35.783685 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcef3b2-0dcc-4539-9f90-ec8bf4e2a039 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:35.828863 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab4f752-eeb8-4dbd-87c7-8e5f8d1070bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:35.829532 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab4f752-eeb8-4dbd-87c7-8e5f8d1070bb None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:35.835248 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab4f752-eeb8-4dbd-87c7-8e5f8d1070bb None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:35.835248 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab4f752-eeb8-4dbd-87c7-8e5f8d1070bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:35.851245 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab4f752-eeb8-4dbd-87c7-8e5f8d1070bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:35.851245 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab4f752-eeb8-4dbd-87c7-8e5f8d1070bb None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:35.868151 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-eab4f752-eeb8-4dbd-87c7-8e5f8d1070bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:35.868151 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:35.868151 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:35.868151 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:35.868151 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:35.868151 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:35.868151 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:35.868151 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:35.868151 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:35.868151 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:35.868151 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:35.868151 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:35.868151 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:30:35.868151 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:30:35.868151 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:30:35.868151 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:30:35.869765 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:35.869765 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:35.869765 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:35.869765 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:35.869765 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:35.869765 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:35.869765 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:35.869765 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:35.869765 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:35.869765 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:35.869765 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:35.869765 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:35.869765 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:35.869765 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:35.869765 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:35.869765 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:35.869765 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:30:35.870686 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:30:35.870686 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:35.870686 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:35.876623 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 484/967] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:35.879414 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab4f752-eeb8-4dbd-87c7-8e5f8d1070bb None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:35.880257 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:35.902684 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:35.912913 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fcc7ee-9048-45a8-827c-b8bbf020d25e None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:35.960121 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d3869e-6cae-4a7d-af84-cc939b1af7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:35.961043 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3869e-6cae-4a7d-af84-cc939b1af7e8 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:35.961655 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3869e-6cae-4a7d-af84-cc939b1af7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:35.962088 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3869e-6cae-4a7d-af84-cc939b1af7e8 None admin] PATH_INFO: `/v3/services` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:35.963904 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d3869e-6cae-4a7d-af84-cc939b1af7e8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:35.964548 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d3869e-6cae-4a7d-af84-cc939b1af7e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:35.966217 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d3869e-6cae-4a7d-af84-cc939b1af7e8 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:35.977108 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 484/968] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Aug 30 14:30:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:35.978513 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d3869e-6cae-4a7d-af84-cc939b1af7e8 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:36.049234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab4f752-eeb8-4dbd-87c7-8e5f8d1070bb None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:36.108863 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce0dfd1-f679-499e-9d54-c22a5f28db6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:36.109412 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce0dfd1-f679-499e-9d54-c22a5f28db6f None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:36.109695 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce0dfd1-f679-499e-9d54-c22a5f28db6f None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:36.110053 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce0dfd1-f679-499e-9d54-c22a5f28db6f None admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:36.110620 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce0dfd1-f679-499e-9d54-c22a5f28db6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:36.110925 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce0dfd1-f679-499e-9d54-c22a5f28db6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:36.112327 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce0dfd1-f679-499e-9d54-c22a5f28db6f None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:36.117629 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d3869e-6cae-4a7d-af84-cc939b1af7e8 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:36.137281 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 485/969] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:30:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:36.138682 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce0dfd1-f679-499e-9d54-c22a5f28db6f None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:36.166140 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26132fd7-9611-4e07-8429-4cf5122a882a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:36.166984 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26132fd7-9611-4e07-8429-4cf5122a882a None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:36.167457 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26132fd7-9611-4e07-8429-4cf5122a882a None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:36.168742 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26132fd7-9611-4e07-8429-4cf5122a882a None admin] PATH_INFO: `/v3/services` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:36.169607 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26132fd7-9611-4e07-8429-4cf5122a882a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:36.170068 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26132fd7-9611-4e07-8429-4cf5122a882a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:36.172539 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26132fd7-9611-4e07-8429-4cf5122a882a None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:36.174061 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:36.191333 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 485/970] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Aug 30 14:30:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:36.192416 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26132fd7-9611-4e07-8429-4cf5122a882a None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:36.248163 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:36.272944 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce0dfd1-f679-499e-9d54-c22a5f28db6f None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:36.331685 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7bd686-83ee-4673-afd5-54144fd55337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:36.331685 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7bd686-83ee-4673-afd5-54144fd55337 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:36.334090 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7bd686-83ee-4673-afd5-54144fd55337 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:36.334090 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7bd686-83ee-4673-afd5-54144fd55337 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:36.345195 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7bd686-83ee-4673-afd5-54144fd55337 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:36.348630 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26132fd7-9611-4e07-8429-4cf5122a882a None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:36.350157 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7bd686-83ee-4673-afd5-54144fd55337 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:36.366690 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-8c7bd686-83ee-4673-afd5-54144fd55337 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:30:36.366690 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:36.366690 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:36.366690 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:36.366690 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:36.366690 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:36.366690 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:36.366690 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:36.366690 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:36.366690 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:36.366690 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:36.366690 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:36.366690 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:30:36.366690 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:30:36.366690 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:30:36.367658 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:30:36.367658 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:36.367658 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:36.367658 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:36.367658 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:36.367658 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:36.367658 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:36.367658 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:36.367658 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:36.367658 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:36.367658 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:36.367658 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:36.367658 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:36.367658 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:36.367658 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:36.367658 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:36.367658 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:36.368567 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:30:36.368567 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:30:36.368567 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:30:36.368567 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:30:36.368567 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:30:36.368567 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:30:36.368567 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:30:36.368567 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:36.373746 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 486/971] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:30:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:36.374623 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7bd686-83ee-4673-afd5-54144fd55337 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:36.408149 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1255be01-c15c-4937-bff8-d862bfb434cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:36.408782 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255be01-c15c-4937-bff8-d862bfb434cf None admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:36.409063 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255be01-c15c-4937-bff8-d862bfb434cf None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:36.409338 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255be01-c15c-4937-bff8-d862bfb434cf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:36.410022 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1255be01-c15c-4937-bff8-d862bfb434cf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:36.411964 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1255be01-c15c-4937-bff8-d862bfb434cf None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:36.446583 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 486/972] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:30:36 2022] POST /identity/v3/endpoints => generated 349 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:30:36.561000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7bd686-83ee-4673-afd5-54144fd55337 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:36.614615 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad568a-1eb7-4910-b09c-907802ae7f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:36.615699 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad568a-1eb7-4910-b09c-907802ae7f5a None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:36.616202 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad568a-1eb7-4910-b09c-907802ae7f5a None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:36.616690 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad568a-1eb7-4910-b09c-907802ae7f5a None admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:36.617616 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dad568a-1eb7-4910-b09c-907802ae7f5a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:36.618126 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dad568a-1eb7-4910-b09c-907802ae7f5a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:36.619874 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dad568a-1eb7-4910-b09c-907802ae7f5a None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:36.621984 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:36.637101 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 487/973] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 30 14:30:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:36.644944 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1255be01-c15c-4937-bff8-d862bfb434cf None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:36.721576 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:36.726952 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e37266-3424-45dd-8d98-5ec3aa466ec1 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:36.727344 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e37266-3424-45dd-8d98-5ec3aa466ec1 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:36.733995 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e37266-3424-45dd-8d98-5ec3aa466ec1 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:36.733995 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 488/974] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:30:36.737107 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b317c1-2493-424c-ae9e-90b430e80ac7 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:36.737275 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b317c1-2493-424c-ae9e-90b430e80ac7 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:36.738406 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b317c1-2493-424c-ae9e-90b430e80ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:36.763296 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-09b317c1-2493-424c-ae9e-90b430e80ac7 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:36.766440 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-09b317c1-2493-424c-ae9e-90b430e80ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:36.789970 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1255be01-c15c-4937-bff8-d862bfb434cf None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:36.845164 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bfcb6c-ed78-446a-96d7-564eccccb853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:36.846056 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bfcb6c-ed78-446a-96d7-564eccccb853 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:36.846489 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bfcb6c-ed78-446a-96d7-564eccccb853 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:36.847071 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bfcb6c-ed78-446a-96d7-564eccccb853 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:36.848272 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bfcb6c-ed78-446a-96d7-564eccccb853 None admin] RBAC: Adding query filter params (role.id=2345dd1fbf154d6888e8113c51914966, scope.project.id=01cf71ca5abf4576b89d10c85af6d2de, user.id=f60bee471e954932a928ab2cf7d66c82) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:36.848735 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bfcb6c-ed78-446a-96d7-564eccccb853 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:36.850340 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bfcb6c-ed78-446a-96d7-564eccccb853 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:36.859460 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 487/975] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:30:36 2022] GET /identity/v3/role_assignments?user.id=f60bee471e954932a928ab2cf7d66c82&scope.project.id=01cf71ca5abf4576b89d10c85af6d2de&role.id=2345dd1fbf154d6888e8113c51914966 => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:36.898357 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 489/976] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:36 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:30:37.205853 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bfcb6c-ed78-446a-96d7-564eccccb853 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:37.329441 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bfcb6c-ed78-446a-96d7-564eccccb853 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:37.355584 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:37.371883 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3066cae0-d62b-4f39-9762-89a05aa71ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:37.372991 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066cae0-d62b-4f39-9762-89a05aa71ad0 None admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:37.373457 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066cae0-d62b-4f39-9762-89a05aa71ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:37.373920 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066cae0-d62b-4f39-9762-89a05aa71ad0 None admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:37.374876 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3066cae0-d62b-4f39-9762-89a05aa71ad0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:37.376764 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3066cae0-d62b-4f39-9762-89a05aa71ad0 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:37.380562 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: WARNING keystone.server.flask.common [None req-3066cae0-d62b-4f39-9762-89a05aa71ad0 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 30 14:30:37.395561 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.sql.core [None req-3066cae0-d62b-4f39-9762-89a05aa71ad0 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 30 14:30:37.395561 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 30 14:30:37.395561 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [parameters: {'user_id': 'd2b51fdea0834b9895f48f8ef6026073', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 30 14:30:37.395561 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81689) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 30 14:30:37.405099 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-3066cae0-d62b-4f39-9762-89a05aa71ad0 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 30 14:30:37.405099 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:37.405099 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 14:30:37.405099 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 14:30:37.405099 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 14:30:37.405099 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 14:30:37.405099 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 14:30:37.405099 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 14:30:37.405099 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 14:30:37.405099 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application conn.query(q) Aug 30 14:30:37.405099 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 14:30:37.405099 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 14:30:37.405099 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 14:30:37.405099 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application result.read() Aug 30 14:30:37.405099 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 14:30:37.406011 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 14:30:37.406011 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 14:30:37.406011 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 14:30:37.406011 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 14:30:37.406011 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 14:30:37.406011 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 14:30:37.406011 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 14:30:37.406011 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 30 14:30:37.406011 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:37.406011 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 30 14:30:37.406011 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:37.406011 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:37.406011 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 30 14:30:37.406011 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 30 14:30:37.406011 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 30 14:30:37.406011 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 30 14:30:37.406011 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 14:30:37.406011 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application next(self.gen) Aug 30 14:30:37.406953 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 30 14:30:37.406953 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application yield resource Aug 30 14:30:37.406953 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 14:30:37.406953 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application next(self.gen) Aug 30 14:30:37.406953 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 30 14:30:37.406953 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application self.session.rollback() Aug 30 14:30:37.406953 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 30 14:30:37.406953 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application self.force_reraise() Aug 30 14:30:37.406953 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 30 14:30:37.406953 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise self.value Aug 30 14:30:37.406953 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 30 14:30:37.406953 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 30 14:30:37.406953 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 30 14:30:37.406953 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application session.commit() Aug 30 14:30:37.406953 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 30 14:30:37.406953 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 30 14:30:37.406953 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 30 14:30:37.407900 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application self._prepare_impl() Aug 30 14:30:37.407900 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 30 14:30:37.407900 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application self.session.flush() Aug 30 14:30:37.407900 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 30 14:30:37.407900 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application self._flush(objects) Aug 30 14:30:37.407900 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 30 14:30:37.407900 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 30 14:30:37.407900 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 30 14:30:37.407900 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application compat.raise_( Aug 30 14:30:37.407900 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 14:30:37.407900 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception Aug 30 14:30:37.407900 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 30 14:30:37.407900 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application flush_context.execute() Aug 30 14:30:37.407900 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 30 14:30:37.407900 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rec.execute(self) Aug 30 14:30:37.407900 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 30 14:30:37.407900 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 30 14:30:37.408786 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 30 14:30:37.408786 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 30 14:30:37.408786 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 30 14:30:37.408786 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 30 14:30:37.408786 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 30 14:30:37.408786 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 30 14:30:37.408786 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 30 14:30:37.408786 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 30 14:30:37.408786 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 30 14:30:37.408786 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 30 14:30:37.408786 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 30 14:30:37.408786 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 30 14:30:37.408786 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 30 14:30:37.408786 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 30 14:30:37.408786 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 14:30:37.408786 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception Aug 30 14:30:37.409636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 14:30:37.409636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 14:30:37.409636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 14:30:37.409636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 14:30:37.409636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 14:30:37.409636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 14:30:37.409636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 14:30:37.409636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application conn.query(q) Aug 30 14:30:37.409636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 14:30:37.409636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 14:30:37.409636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 14:30:37.409636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application result.read() Aug 30 14:30:37.409636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 14:30:37.409636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 14:30:37.409636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 14:30:37.409636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 14:30:37.410552 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 14:30:37.410552 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 14:30:37.410552 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 14:30:37.410552 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 14:30:37.410552 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 30 14:30:37.410552 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 30 14:30:37.410552 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [parameters: {'user_id': 'd2b51fdea0834b9895f48f8ef6026073', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 30 14:30:37.410552 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 30 14:30:37.410552 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:37.410552 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:30:37.410552 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:37.410552 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:37.410552 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:37.410552 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:37.410552 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:37.411680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:37.411680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:37.411680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:37.411680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:37.411680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:37.411680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:37.411680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:37.411680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 30 14:30:37.411680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 30 14:30:37.411680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:37.411680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:37.411680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:30:37.411680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:30:37.411680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:30:37.411680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:30:37.411680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 30 14:30:37.411680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 30 14:30:37.412728 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 30 14:30:37.412728 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 30 14:30:37.412728 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 30 14:30:37.412728 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 30 14:30:37.412728 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 30 14:30:37.412728 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:37.414695 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 488/977] 10.222.0.30 () {60 vars in 1131 bytes} [Tue Aug 30 14:30:37 2022] POST /identity/v3/users => generated 160 bytes in 210 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:37.428408 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b317c1-2493-424c-ae9e-90b430e80ac7 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:37.580293 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b317c1-2493-424c-ae9e-90b430e80ac7 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:37.624551 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:37.625382 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cdf15d-6e05-46fe-a815-1e52390fce4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:37.626196 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cdf15d-6e05-46fe-a815-1e52390fce4b None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:37.626624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cdf15d-6e05-46fe-a815-1e52390fce4b None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:37.627084 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cdf15d-6e05-46fe-a815-1e52390fce4b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:37.637871 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cdf15d-6e05-46fe-a815-1e52390fce4b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:37.639559 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cdf15d-6e05-46fe-a815-1e52390fce4b None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:37.652950 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-11cdf15d-6e05-46fe-a815-1e52390fce4b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 14:30:37.652950 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:37.652950 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:37.652950 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:37.652950 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:37.652950 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:37.652950 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:37.652950 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:37.652950 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:37.652950 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:37.652950 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:37.652950 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:37.652950 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:30:37.652950 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:30:37.652950 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:30:37.652950 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:30:37.654166 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:37.654166 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:37.654166 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:30:37.654166 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:30:37.654166 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:30:37.654166 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:30:37.654166 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:37.654166 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:37.654166 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:37.654166 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:37.654166 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:37.654166 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:37.654166 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:37.654166 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:37.654166 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:37.654166 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:37.654166 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:37.654166 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:37.655197 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:37.655197 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:37.655197 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:30:37.655197 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:30:37.655197 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:30:37.655197 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:30:37.655197 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:30:37.655197 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:30:37.655197 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 14:30:37.655197 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:37.657679 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 490/978] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:30:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:37.660814 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb27d0-4b70-4588-8d79-a2b627404084 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:37.661101 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb27d0-4b70-4588-8d79-a2b627404084 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:37.661369 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb27d0-4b70-4588-8d79-a2b627404084 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:37.664252 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cdf15d-6e05-46fe-a815-1e52390fce4b None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:37.682969 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 489/979] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:37 2022] GET /identity => generated 270 bytes in 23 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:37.685765 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447dc2c-d0e2-4fd8-9c05-39884c65b33b None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:37.686054 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447dc2c-d0e2-4fd8-9c05-39884c65b33b None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:37.686333 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447dc2c-d0e2-4fd8-9c05-39884c65b33b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:37.728908 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-0447dc2c-d0e2-4fd8-9c05-39884c65b33b None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:37.730506 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-0447dc2c-d0e2-4fd8-9c05-39884c65b33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:37.801060 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cdf15d-6e05-46fe-a815-1e52390fce4b None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:37.868376 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bf606c-95bf-4b65-a43b-86717a6221a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:37.868376 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf606c-95bf-4b65-a43b-86717a6221a1 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:37.868376 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf606c-95bf-4b65-a43b-86717a6221a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:37.868750 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf606c-95bf-4b65-a43b-86717a6221a1 None admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:37.873967 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bf606c-95bf-4b65-a43b-86717a6221a1 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:37.873967 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bf606c-95bf-4b65-a43b-86717a6221a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:37.873967 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bf606c-95bf-4b65-a43b-86717a6221a1 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:37.874449 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 490/980] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:30:37.953533 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 491/981] 10.222.0.30 () {56 vars in 1101 bytes} [Tue Aug 30 14:30:37 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:30:38.191788 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0447dc2c-d0e2-4fd8-9c05-39884c65b33b None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:38.324568 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0447dc2c-d0e2-4fd8-9c05-39884c65b33b None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:38.336207 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:38.371239 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bbe540-f65d-404d-932b-bf201b56c82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:38.371239 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bbe540-f65d-404d-932b-bf201b56c82c None admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:38.371239 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bbe540-f65d-404d-932b-bf201b56c82c None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:38.371239 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bbe540-f65d-404d-932b-bf201b56c82c None admin] PATH_INFO: `/v3/services` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:38.371671 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bbe540-f65d-404d-932b-bf201b56c82c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:38.371671 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bbe540-f65d-404d-932b-bf201b56c82c None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:38.390803 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 491/982] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Aug 30 14:30:38 2022] POST /identity/v3/services => generated 242 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:30:38.647874 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a79d1e-25b2-4058-9173-204155d778c8 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:38.647874 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a79d1e-25b2-4058-9173-204155d778c8 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:38.647874 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a79d1e-25b2-4058-9173-204155d778c8 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:38.648817 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 492/983] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:38.655079 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5336237-a4a0-46b5-8e72-ceb1b1474e25 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:38.655397 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5336237-a4a0-46b5-8e72-ceb1b1474e25 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:38.655675 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5336237-a4a0-46b5-8e72-ceb1b1474e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:38.675384 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-d5336237-a4a0-46b5-8e72-ceb1b1474e25 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:38.676580 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-d5336237-a4a0-46b5-8e72-ceb1b1474e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:38.758600 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 492/984] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:38 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:30:39.042073 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a79d1e-25b2-4058-9173-204155d778c8 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:39.091900 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:39.095140 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d041b3-8d88-401a-9e4f-88a2a623db00 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:39.095456 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d041b3-8d88-401a-9e4f-88a2a623db00 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:39.095729 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d041b3-8d88-401a-9e4f-88a2a623db00 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:39.097254 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 493/985] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:39.102332 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c384999-d247-4707-b0b1-1374220721aa None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:39.102616 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c384999-d247-4707-b0b1-1374220721aa None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:39.102891 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c384999-d247-4707-b0b1-1374220721aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:39.122003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-9c384999-d247-4707-b0b1-1374220721aa None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:39.123199 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-9c384999-d247-4707-b0b1-1374220721aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:39.164271 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a79d1e-25b2-4058-9173-204155d778c8 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:39.205475 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa50171c-782f-4916-be26-5fd10562a5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:39.206507 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50171c-782f-4916-be26-5fd10562a5ea None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:39.206942 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50171c-782f-4916-be26-5fd10562a5ea None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:39.207471 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50171c-782f-4916-be26-5fd10562a5ea None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:39.208214 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa50171c-782f-4916-be26-5fd10562a5ea None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:39.209735 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa50171c-782f-4916-be26-5fd10562a5ea None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:39.211482 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 494/986] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:39 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:30:39.223177 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-fa50171c-782f-4916-be26-5fd10562a5ea None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 14:30:39.223177 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:39.223177 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:30:39.223177 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:30:39.223177 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:30:39.223177 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:30:39.223177 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:30:39.223177 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:30:39.223177 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 14:30:39.223177 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:39.223177 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:30:39.223177 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:39.223177 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:39.223177 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:39.223177 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:39.223177 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:39.223177 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:39.224187 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:39.224187 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:39.224187 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:39.224187 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:39.224187 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:39.224187 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:39.224187 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:30:39.224187 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:30:39.224187 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:30:39.224187 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:30:39.224187 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:39.224187 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:39.224187 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:39.224187 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:39.224187 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:39.224187 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:39.225029 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:39.225029 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:39.225029 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:39.225029 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:39.225029 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:39.225029 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:39.225029 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:39.225029 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:39.225029 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:39.225029 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:39.225029 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:30:39.225029 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:30:39.225029 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 14:30:39.225029 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:39.230492 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 493/987] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Aug 30 14:30:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:39.231005 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c384999-d247-4707-b0b1-1374220721aa None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:39.263582 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:39.353244 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c384999-d247-4707-b0b1-1374220721aa None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:39.391139 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291982ac-213c-42a0-bfa4-499dbf28c535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:39.392320 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291982ac-213c-42a0-bfa4-499dbf28c535 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:39.392798 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291982ac-213c-42a0-bfa4-499dbf28c535 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:39.393278 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291982ac-213c-42a0-bfa4-499dbf28c535 None admin] PATH_INFO: `/v3/services` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:39.394102 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291982ac-213c-42a0-bfa4-499dbf28c535 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:39.394686 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291982ac-213c-42a0-bfa4-499dbf28c535 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:39.396429 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291982ac-213c-42a0-bfa4-499dbf28c535 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:39.402508 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 495/988] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Aug 30 14:30:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:39.407892 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa50171c-782f-4916-be26-5fd10562a5ea None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:39.495551 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291982ac-213c-42a0-bfa4-499dbf28c535 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:39.523321 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa50171c-782f-4916-be26-5fd10562a5ea None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:39.533186 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:39.567866 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c19502-89cd-4b0c-9436-080653f2002b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:39.568737 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c19502-89cd-4b0c-9436-080653f2002b None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:39.569161 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c19502-89cd-4b0c-9436-080653f2002b None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:39.569569 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c19502-89cd-4b0c-9436-080653f2002b None admin] PATH_INFO: `/v3/services` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:39.570334 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c19502-89cd-4b0c-9436-080653f2002b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:39.570957 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c19502-89cd-4b0c-9436-080653f2002b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:39.572708 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c19502-89cd-4b0c-9436-080653f2002b None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:39.579280 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 494/989] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Aug 30 14:30:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:39.584096 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c19502-89cd-4b0c-9436-080653f2002b None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:39.612510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:39.623302 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291982ac-213c-42a0-bfa4-499dbf28c535 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:39.668953 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38632d4-4e3e-4c15-aca5-b989749ff757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:39.669592 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38632d4-4e3e-4c15-aca5-b989749ff757 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:39.669870 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38632d4-4e3e-4c15-aca5-b989749ff757 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:39.670149 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38632d4-4e3e-4c15-aca5-b989749ff757 None admin] PATH_INFO: `/v3/services/image` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:39.670787 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38632d4-4e3e-4c15-aca5-b989749ff757 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:39.672639 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38632d4-4e3e-4c15-aca5-b989749ff757 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:39.684645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-d38632d4-4e3e-4c15-aca5-b989749ff757 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 14:30:39.684645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:39.684645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:30:39.684645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:30:39.684645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:30:39.684645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:30:39.684645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:30:39.684645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:30:39.684645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 14:30:39.684645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:39.684645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:30:39.684645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:39.684645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:39.684645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:39.684645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:39.684645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:39.684645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:39.685674 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:39.685674 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:39.685674 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:39.685674 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:39.685674 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:39.685674 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:39.685674 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:30:39.685674 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:30:39.685674 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:30:39.685674 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:30:39.685674 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:39.685674 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:39.685674 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:39.685674 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:39.685674 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:39.685674 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:39.687021 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:39.687021 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:39.687021 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:39.687021 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:39.687021 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:39.687021 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:39.687021 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:39.687021 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:39.687021 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:39.687021 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:39.687021 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:30:39.687021 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:30:39.687021 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 14:30:39.687021 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:39.687021 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 496/990] 10.222.0.30 () {56 vars in 1105 bytes} [Tue Aug 30 14:30:39 2022] GET /identity/v3/services/image => generated 86 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:30:39.692212 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38632d4-4e3e-4c15-aca5-b989749ff757 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:39.710380 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c19502-89cd-4b0c-9436-080653f2002b None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:39.753800 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f07d04-17df-4c57-8618-8d0e4d63a4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:39.754412 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f07d04-17df-4c57-8618-8d0e4d63a4dc None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:39.754686 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f07d04-17df-4c57-8618-8d0e4d63a4dc None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:39.755400 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f07d04-17df-4c57-8618-8d0e4d63a4dc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:39.755775 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f07d04-17df-4c57-8618-8d0e4d63a4dc None admin] RBAC: Adding query filter params (interface=admin, service_id=a45fcd7e95f74c5ab6be5d0379d50311, region_id=RegionOne) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:39.756081 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f07d04-17df-4c57-8618-8d0e4d63a4dc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:39.758281 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f07d04-17df-4c57-8618-8d0e4d63a4dc None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:39.764034 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 495/991] 10.222.0.30 () {56 vars in 1249 bytes} [Tue Aug 30 14:30:39 2022] GET /identity/v3/endpoints?service_id=a45fcd7e95f74c5ab6be5d0379d50311&interface=admin®ion_id=RegionOne => generated 193 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:30:39.769873 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f07d04-17df-4c57-8618-8d0e4d63a4dc None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:39.813428 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:39.823656 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38632d4-4e3e-4c15-aca5-b989749ff757 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:39.869620 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf1ac6-5f8f-42df-905c-bb883c4b7f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:39.870301 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf1ac6-5f8f-42df-905c-bb883c4b7f9f None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:39.871354 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf1ac6-5f8f-42df-905c-bb883c4b7f9f None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:39.871546 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf1ac6-5f8f-42df-905c-bb883c4b7f9f None admin] PATH_INFO: `/v3/services` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:39.872257 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bf1ac6-5f8f-42df-905c-bb883c4b7f9f None admin] RBAC: Adding query filter params (name=image) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:39.872532 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bf1ac6-5f8f-42df-905c-bb883c4b7f9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:39.874360 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bf1ac6-5f8f-42df-905c-bb883c4b7f9f None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:39.881559 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 497/992] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:39 2022] GET /identity/v3/services?name=image => generated 122 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:39.887999 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf1ac6-5f8f-42df-905c-bb883c4b7f9f None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:39.904006 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f07d04-17df-4c57-8618-8d0e4d63a4dc None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:39.909121 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:39.947953 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2cc852-9526-4247-89d0-b633234641ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:39.948853 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2cc852-9526-4247-89d0-b633234641ab None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:39.949139 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2cc852-9526-4247-89d0-b633234641ab None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:39.949412 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2cc852-9526-4247-89d0-b633234641ab None admin] PATH_INFO: `/v3/services` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:39.950025 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2cc852-9526-4247-89d0-b633234641ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:39.951571 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2cc852-9526-4247-89d0-b633234641ab None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:39.958962 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 496/993] 10.222.0.30 () {56 vars in 1087 bytes} [Tue Aug 30 14:30:39 2022] GET /identity/v3/services => generated 1727 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:30:40.010375 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf1ac6-5f8f-42df-905c-bb883c4b7f9f None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:40.053588 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927b39c-37c3-4f7d-9757-729796e189be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:40.054244 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927b39c-37c3-4f7d-9757-729796e189be None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:40.054524 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927b39c-37c3-4f7d-9757-729796e189be None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:40.054792 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927b39c-37c3-4f7d-9757-729796e189be None admin] PATH_INFO: `/v3/services` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:40.055458 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f927b39c-37c3-4f7d-9757-729796e189be None admin] RBAC: Adding query filter params (type=image) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:40.055758 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f927b39c-37c3-4f7d-9757-729796e189be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:40.057443 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f927b39c-37c3-4f7d-9757-729796e189be None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:40.069783 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 498/994] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:39 2022] GET /identity/v3/services?type=image => generated 122 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:40.387379 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e4a75-0e8b-4c2d-94fe-72d21191e1d0 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:40.387724 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e4a75-0e8b-4c2d-94fe-72d21191e1d0 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:40.387938 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e4a75-0e8b-4c2d-94fe-72d21191e1d0 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:40.389627 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 497/995] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:40.394859 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289399c-4061-4a98-abc8-e1947eeac565 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:40.395166 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289399c-4061-4a98-abc8-e1947eeac565 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:40.395470 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289399c-4061-4a98-abc8-e1947eeac565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:40.415224 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-9289399c-4061-4a98-abc8-e1947eeac565 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:40.416796 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-9289399c-4061-4a98-abc8-e1947eeac565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:40.496875 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 499/996] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:40 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:30:40.774051 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238e4a75-0e8b-4c2d-94fe-72d21191e1d0 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:40.879401 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:40.885754 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238e4a75-0e8b-4c2d-94fe-72d21191e1d0 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:40.924080 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cc14ba-3381-4013-bcf8-716a65197abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:40.924714 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc14ba-3381-4013-bcf8-716a65197abc None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:40.924992 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc14ba-3381-4013-bcf8-716a65197abc None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:40.925260 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc14ba-3381-4013-bcf8-716a65197abc None admin] PATH_INFO: `/v3/services/compute` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:40.925838 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cc14ba-3381-4013-bcf8-716a65197abc None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:40.927191 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cc14ba-3381-4013-bcf8-716a65197abc None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:40.937701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-17cc14ba-3381-4013-bcf8-716a65197abc None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 14:30:40.937701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:40.937701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:30:40.937701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:30:40.937701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:30:40.937701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:30:40.937701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:30:40.937701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:30:40.937701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 14:30:40.937701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:40.937701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:30:40.937701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:40.937701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:40.937701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:40.937701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:40.937701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:40.937701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:40.938720 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:40.938720 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:40.938720 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:40.938720 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:40.938720 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:40.938720 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:40.938720 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:30:40.938720 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:30:40.938720 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:30:40.938720 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:30:40.938720 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:40.938720 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:40.938720 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:40.938720 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:40.938720 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:40.938720 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:40.939613 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:40.939613 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:40.939613 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:40.939613 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:40.939613 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:40.939613 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:40.939613 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:40.939613 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:40.939613 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:40.939613 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:40.939613 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:30:40.939613 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:30:40.939613 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 14:30:40.939613 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:40.943558 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 498/997] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:30:40 2022] GET /identity/v3/services/compute => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:40.945023 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9289399c-4061-4a98-abc8-e1947eeac565 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:40.975635 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:41.061509 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9289399c-4061-4a98-abc8-e1947eeac565 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:41.098778 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26bd912-ae78-4813-9765-deb7951ef735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:41.099573 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26bd912-ae78-4813-9765-deb7951ef735 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:41.099846 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26bd912-ae78-4813-9765-deb7951ef735 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:41.100166 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26bd912-ae78-4813-9765-deb7951ef735 None admin] PATH_INFO: `/v3/services` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:41.100836 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26bd912-ae78-4813-9765-deb7951ef735 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:41.101139 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26bd912-ae78-4813-9765-deb7951ef735 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:41.102574 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26bd912-ae78-4813-9765-deb7951ef735 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:41.113465 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 500/998] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Aug 30 14:30:40 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:41.114495 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cc14ba-3381-4013-bcf8-716a65197abc None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:41.229917 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cc14ba-3381-4013-bcf8-716a65197abc None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:41.266502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef69bc-2b44-426f-9e4d-2b89c23e3bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:41.267106 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef69bc-2b44-426f-9e4d-2b89c23e3bc4 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:41.267582 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef69bc-2b44-426f-9e4d-2b89c23e3bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:41.267850 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef69bc-2b44-426f-9e4d-2b89c23e3bc4 None admin] PATH_INFO: `/v3/services` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:41.268477 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef69bc-2b44-426f-9e4d-2b89c23e3bc4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:41.268773 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef69bc-2b44-426f-9e4d-2b89c23e3bc4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:41.270151 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef69bc-2b44-426f-9e4d-2b89c23e3bc4 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:41.280966 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 499/999] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Aug 30 14:30:41 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:30:41.282355 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26bd912-ae78-4813-9765-deb7951ef735 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:41.371762 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:41.399371 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26bd912-ae78-4813-9765-deb7951ef735 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:41.436800 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f43ad8-dd7f-484c-8cda-9ecabee78eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:41.437460 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f43ad8-dd7f-484c-8cda-9ecabee78eef None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:41.437736 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f43ad8-dd7f-484c-8cda-9ecabee78eef None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:41.438049 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f43ad8-dd7f-484c-8cda-9ecabee78eef None admin] PATH_INFO: `/v3/endpoints` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:41.438756 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f43ad8-dd7f-484c-8cda-9ecabee78eef None admin] RBAC: Adding query filter params (interface=public, service_id=e76e4693dcd942858b6ea2064b3814f3, region_id=RegionOne) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:41.439234 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f43ad8-dd7f-484c-8cda-9ecabee78eef None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:41.440629 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f43ad8-dd7f-484c-8cda-9ecabee78eef None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:41.451455 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 501/1000] 10.222.0.30 () {56 vars in 1251 bytes} [Tue Aug 30 14:30:41 2022] GET /identity/v3/endpoints?service_id=e76e4693dcd942858b6ea2064b3814f3&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:41.452534 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef69bc-2b44-426f-9e4d-2b89c23e3bc4 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:41.484493 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:41.569540 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef69bc-2b44-426f-9e4d-2b89c23e3bc4 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:41.607607 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa892d77-7aa7-4d04-9dd5-b76f2540f30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:41.608255 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa892d77-7aa7-4d04-9dd5-b76f2540f30f None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:41.608532 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa892d77-7aa7-4d04-9dd5-b76f2540f30f None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:41.608863 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa892d77-7aa7-4d04-9dd5-b76f2540f30f None admin] PATH_INFO: `/v3/services` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:41.609466 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa892d77-7aa7-4d04-9dd5-b76f2540f30f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:41.610875 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa892d77-7aa7-4d04-9dd5-b76f2540f30f None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:41.623643 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 500/1001] 10.222.0.30 () {56 vars in 1087 bytes} [Tue Aug 30 14:30:41 2022] GET /identity/v3/services => generated 1727 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:30:41.969398 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88869f4-9c6e-4f65-bcec-26b483ed80f3 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:41.969872 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88869f4-9c6e-4f65-bcec-26b483ed80f3 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:41.969872 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88869f4-9c6e-4f65-bcec-26b483ed80f3 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:41.971371 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 502/1002] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:41.976666 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c031a6d-1ad6-4034-b0c9-f914fdd0d1e1 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:41.977081 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c031a6d-1ad6-4034-b0c9-f914fdd0d1e1 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:41.977448 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c031a6d-1ad6-4034-b0c9-f914fdd0d1e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:41.996486 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-2c031a6d-1ad6-4034-b0c9-f914fdd0d1e1 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:41.997811 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-2c031a6d-1ad6-4034-b0c9-f914fdd0d1e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:42.063479 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf5c69-79c9-44e0-9860-c8bf306a544c None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:42.063982 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf5c69-79c9-44e0-9860-c8bf306a544c None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:42.064439 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf5c69-79c9-44e0-9860-c8bf306a544c None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:42.065980 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 503/1003] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:30:42.071080 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f905250-b182-40f2-aecb-8c549f43c730 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:42.071407 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f905250-b182-40f2-aecb-8c549f43c730 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:42.072274 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f905250-b182-40f2-aecb-8c549f43c730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:42.082609 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 501/1004] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:41 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:30:42.092617 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-1f905250-b182-40f2-aecb-8c549f43c730 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:42.093806 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-1f905250-b182-40f2-aecb-8c549f43c730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:42.172904 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 504/1005] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:42 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:30:42.354394 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c031a6d-1ad6-4034-b0c9-f914fdd0d1e1 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:42.377170 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:42.452546 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f905250-b182-40f2-aecb-8c549f43c730 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:42.469750 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c031a6d-1ad6-4034-b0c9-f914fdd0d1e1 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:42.515377 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c21adba-60d2-475a-91d2-111ae084d1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:42.516556 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c21adba-60d2-475a-91d2-111ae084d1fc None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:42.517083 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c21adba-60d2-475a-91d2-111ae084d1fc None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:42.517610 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c21adba-60d2-475a-91d2-111ae084d1fc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:42.518402 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c21adba-60d2-475a-91d2-111ae084d1fc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:42.520115 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c21adba-60d2-475a-91d2-111ae084d1fc None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:42.533449 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-3c21adba-60d2-475a-91d2-111ae084d1fc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 14:30:42.533449 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:42.533449 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:30:42.533449 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:30:42.533449 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:30:42.533449 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:30:42.533449 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:30:42.533449 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:30:42.533449 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 14:30:42.533449 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:42.533449 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:30:42.533449 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:42.533449 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:42.533449 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:42.533449 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:42.533449 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:42.533449 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:42.534502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:42.534502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:42.534502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:42.534502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:42.534502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:42.534502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:42.534502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:30:42.534502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:30:42.534502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:30:42.534502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:30:42.534502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:42.534502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:42.534502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:42.534502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:42.534502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:42.534502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:42.535376 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:42.535376 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:42.535376 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:42.535376 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:42.535376 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:42.535376 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:42.535376 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:42.535376 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:42.535376 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:42.535376 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:42.535376 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:30:42.535376 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:30:42.535376 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 14:30:42.535376 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:42.543075 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:42.544261 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 502/1006] 10.222.0.30 () {56 vars in 1126 bytes} [Tue Aug 30 14:30:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:42.545426 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c21adba-60d2-475a-91d2-111ae084d1fc None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:42.586462 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f905250-b182-40f2-aecb-8c549f43c730 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:42.632676 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb705849-fb96-4fc9-9a78-ad14eb3e726e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:42.633220 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb705849-fb96-4fc9-9a78-ad14eb3e726e None admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:42.633489 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb705849-fb96-4fc9-9a78-ad14eb3e726e None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:42.633765 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb705849-fb96-4fc9-9a78-ad14eb3e726e None admin] PATH_INFO: `/v3/services` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:42.634967 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb705849-fb96-4fc9-9a78-ad14eb3e726e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:42.636394 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb705849-fb96-4fc9-9a78-ad14eb3e726e None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:42.667279 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 505/1007] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Aug 30 14:30:42 2022] POST /identity/v3/services => generated 242 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:30:42.682452 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:42.725712 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c21adba-60d2-475a-91d2-111ae084d1fc None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:42.781692 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffec034-6eec-4528-99b5-dce311de4f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:42.782536 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffec034-6eec-4528-99b5-dce311de4f16 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:42.782898 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffec034-6eec-4528-99b5-dce311de4f16 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:42.783334 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffec034-6eec-4528-99b5-dce311de4f16 None admin] PATH_INFO: `/v3/services` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:42.784047 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffec034-6eec-4528-99b5-dce311de4f16 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:42.784431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffec034-6eec-4528-99b5-dce311de4f16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:42.786046 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffec034-6eec-4528-99b5-dce311de4f16 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:42.799154 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 503/1008] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Aug 30 14:30:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:42.800676 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb705849-fb96-4fc9-9a78-ad14eb3e726e None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:42.939708 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb705849-fb96-4fc9-9a78-ad14eb3e726e None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:42.981328 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:42.990550 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedb34e6-f231-49fc-8d58-dfe2cae2a496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:42.991173 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb34e6-f231-49fc-8d58-dfe2cae2a496 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:42.991480 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb34e6-f231-49fc-8d58-dfe2cae2a496 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:42.991750 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb34e6-f231-49fc-8d58-dfe2cae2a496 None admin] PATH_INFO: `/v3/services` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:42.992389 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedb34e6-f231-49fc-8d58-dfe2cae2a496 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:42.992698 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedb34e6-f231-49fc-8d58-dfe2cae2a496 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:42.994804 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedb34e6-f231-49fc-8d58-dfe2cae2a496 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:43.003557 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 506/1009] 10.222.0.30 () {56 vars in 1122 bytes} [Tue Aug 30 14:30:42 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:43.006011 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffec034-6eec-4528-99b5-dce311de4f16 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:43.114829 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffec034-6eec-4528-99b5-dce311de4f16 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:43.151581 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee77a5d-2ebb-40a5-b1ba-2707b103f008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:43.152208 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee77a5d-2ebb-40a5-b1ba-2707b103f008 None admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:43.152487 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee77a5d-2ebb-40a5-b1ba-2707b103f008 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:43.152762 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee77a5d-2ebb-40a5-b1ba-2707b103f008 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:43.153447 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee77a5d-2ebb-40a5-b1ba-2707b103f008 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:43.154826 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee77a5d-2ebb-40a5-b1ba-2707b103f008 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:43.179509 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 504/1010] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:30:43 2022] POST /identity/v3/endpoints => generated 325 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:30:43.623243 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d24ce9-0590-47f9-b2ad-3407c7453872 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:43.623243 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d24ce9-0590-47f9-b2ad-3407c7453872 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:43.623243 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d24ce9-0590-47f9-b2ad-3407c7453872 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:43.624972 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 507/1011] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:43.629886 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7343c42-346a-4528-b60b-43e83c0cd34e None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:43.630286 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7343c42-346a-4528-b60b-43e83c0cd34e None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:43.630649 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7343c42-346a-4528-b60b-43e83c0cd34e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:43.645195 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:43.669555 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-a7343c42-346a-4528-b60b-43e83c0cd34e None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:43.670760 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-a7343c42-346a-4528-b60b-43e83c0cd34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:43.807314 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 505/1012] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:43 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:30:44.083263 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d24ce9-0590-47f9-b2ad-3407c7453872 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:44.187702 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d24ce9-0590-47f9-b2ad-3407c7453872 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:44.231907 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f0ba0-9571-46d7-9388-ddd938ac201e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:44.232546 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f0ba0-9571-46d7-9388-ddd938ac201e None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:44.232828 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f0ba0-9571-46d7-9388-ddd938ac201e None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:44.233103 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f0ba0-9571-46d7-9388-ddd938ac201e None admin] PATH_INFO: `/v3/services/compute` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:44.233688 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f0ba0-9571-46d7-9388-ddd938ac201e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:44.235055 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f0ba0-9571-46d7-9388-ddd938ac201e None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:44.244586 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-3d5f0ba0-9571-46d7-9388-ddd938ac201e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 14:30:44.244586 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:44.244586 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:30:44.244586 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:30:44.244586 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:30:44.244586 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:30:44.244586 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:30:44.244586 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:30:44.244586 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 14:30:44.244586 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:44.244586 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:30:44.244586 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:44.244586 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:44.244586 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:44.244586 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:44.244586 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:44.244586 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:44.245609 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:44.245609 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:44.245609 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:44.245609 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:44.245609 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:44.245609 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:44.245609 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:30:44.245609 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:30:44.245609 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:30:44.245609 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:30:44.245609 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:44.245609 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:44.245609 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:44.245609 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:44.245609 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:44.245609 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:44.246467 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:44.246467 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:44.246467 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:44.246467 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:44.246467 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:44.246467 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:44.246467 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:44.246467 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:44.246467 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:44.246467 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:44.246467 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:30:44.246467 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:30:44.246467 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 14:30:44.246467 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:44.250630 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 508/1013] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:30:44 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:44.251660 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7343c42-346a-4528-b60b-43e83c0cd34e None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:44.361301 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7343c42-346a-4528-b60b-43e83c0cd34e None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:44.393301 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:44.403098 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eee34c-ada8-4f94-a6ed-6f36db46c073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:44.403747 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eee34c-ada8-4f94-a6ed-6f36db46c073 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:44.404032 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eee34c-ada8-4f94-a6ed-6f36db46c073 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:44.404305 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eee34c-ada8-4f94-a6ed-6f36db46c073 None admin] PATH_INFO: `/v3/services` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:44.404934 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eee34c-ada8-4f94-a6ed-6f36db46c073 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:44.405236 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eee34c-ada8-4f94-a6ed-6f36db46c073 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:44.406605 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eee34c-ada8-4f94-a6ed-6f36db46c073 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:44.416818 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 506/1014] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Aug 30 14:30:44 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:44.417854 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f0ba0-9571-46d7-9388-ddd938ac201e None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:44.480243 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:44.522875 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f0ba0-9571-46d7-9388-ddd938ac201e None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:44.558492 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a3ab1-304b-4710-b1d8-43e24db5535b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:44.559438 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a3ab1-304b-4710-b1d8-43e24db5535b None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:44.559720 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a3ab1-304b-4710-b1d8-43e24db5535b None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:44.559990 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a3ab1-304b-4710-b1d8-43e24db5535b None admin] PATH_INFO: `/v3/services` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:44.560730 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5a3ab1-304b-4710-b1d8-43e24db5535b None admin] RBAC: Adding query filter params (type=compute) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:44.561042 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5a3ab1-304b-4710-b1d8-43e24db5535b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:44.562409 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5a3ab1-304b-4710-b1d8-43e24db5535b None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:44.571574 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 509/1015] 10.222.0.30 () {56 vars in 1112 bytes} [Tue Aug 30 14:30:44 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:30:44.573182 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eee34c-ada8-4f94-a6ed-6f36db46c073 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:44.677612 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eee34c-ada8-4f94-a6ed-6f36db46c073 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:44.705768 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751c871-8a6c-44c4-b273-0a2fbf6d376e None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:44.706076 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751c871-8a6c-44c4-b273-0a2fbf6d376e None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:44.706315 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751c871-8a6c-44c4-b273-0a2fbf6d376e None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:44.707842 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 510/1016] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:30:44.713142 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a317a6e8-b64a-44d9-9de2-f3f89c619ee6 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:44.713490 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a317a6e8-b64a-44d9-9de2-f3f89c619ee6 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:44.713755 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d51c0a-c29f-4b6a-b9c4-659c93576774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:44.713873 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a317a6e8-b64a-44d9-9de2-f3f89c619ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:44.714365 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d51c0a-c29f-4b6a-b9c4-659c93576774 None admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:44.714644 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d51c0a-c29f-4b6a-b9c4-659c93576774 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:44.714914 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d51c0a-c29f-4b6a-b9c4-659c93576774 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:44.715619 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d51c0a-c29f-4b6a-b9c4-659c93576774 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:44.716975 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d51c0a-c29f-4b6a-b9c4-659c93576774 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:44.733420 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-a317a6e8-b64a-44d9-9de2-f3f89c619ee6 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:44.734642 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-a317a6e8-b64a-44d9-9de2-f3f89c619ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:44.739237 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 507/1017] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:30:44 2022] POST /identity/v3/endpoints => generated 334 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:30:44.817416 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 511/1018] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:44 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:30:45.097468 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d51c0a-c29f-4b6a-b9c4-659c93576774 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:45.152342 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:45.208602 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d51c0a-c29f-4b6a-b9c4-659c93576774 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:45.216890 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7132bc0-60dd-4a86-b713-8f8cd5163c30 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:45.217360 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7132bc0-60dd-4a86-b713-8f8cd5163c30 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:45.217449 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7132bc0-60dd-4a86-b713-8f8cd5163c30 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:45.219876 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 512/1019] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:30:45.226452 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d57dd-2981-4b8f-b478-dee40b3253e6 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:45.227370 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d57dd-2981-4b8f-b478-dee40b3253e6 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:45.227370 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d57dd-2981-4b8f-b478-dee40b3253e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:45.250083 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-e76d57dd-2981-4b8f-b478-dee40b3253e6 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:45.251410 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-e76d57dd-2981-4b8f-b478-dee40b3253e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:45.252019 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04037434-3194-4b6a-a67a-2a870245bb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:45.252653 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04037434-3194-4b6a-a67a-2a870245bb6b None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:45.252930 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04037434-3194-4b6a-a67a-2a870245bb6b None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:45.253198 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04037434-3194-4b6a-a67a-2a870245bb6b None admin] PATH_INFO: `/v3/services/image` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:45.253935 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04037434-3194-4b6a-a67a-2a870245bb6b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:45.255317 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04037434-3194-4b6a-a67a-2a870245bb6b None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:45.265773 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-04037434-3194-4b6a-a67a-2a870245bb6b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 14:30:45.265773 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:45.265773 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:30:45.265773 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:30:45.265773 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:30:45.265773 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:30:45.265773 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:30:45.265773 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:30:45.265773 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 14:30:45.265773 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:45.265773 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:30:45.265773 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:45.265773 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:45.265773 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:45.265773 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:45.265773 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:45.265773 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:45.271748 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:45.271748 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:45.271748 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:45.271748 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:45.271748 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:45.271748 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:45.271748 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:30:45.271748 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:30:45.271748 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:30:45.271748 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:30:45.271748 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:45.271748 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:45.271748 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:45.271748 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:45.271748 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:45.271748 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:45.272663 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:45.272663 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:45.272663 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:45.272663 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:45.272663 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:45.272663 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:45.272663 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:45.272663 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:45.272663 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:45.272663 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:45.272663 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:30:45.272663 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:30:45.272663 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 14:30:45.272663 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:45.272663 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 508/1020] 10.222.0.30 () {56 vars in 1105 bytes} [Tue Aug 30 14:30:45 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:45.273527 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04037434-3194-4b6a-a67a-2a870245bb6b None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:45.304797 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:45.335494 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 513/1021] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:45 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:30:45.395109 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04037434-3194-4b6a-a67a-2a870245bb6b None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:45.409606 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:45.431629 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a4a557-4b7d-44b7-a201-6a488c7f34b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:45.432199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4a557-4b7d-44b7-a201-6a488c7f34b1 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:45.432471 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4a557-4b7d-44b7-a201-6a488c7f34b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:45.432737 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4a557-4b7d-44b7-a201-6a488c7f34b1 None admin] PATH_INFO: `/v3/services` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:45.433367 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a4a557-4b7d-44b7-a201-6a488c7f34b1 None admin] RBAC: Adding query filter params (name=image) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:45.433666 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a4a557-4b7d-44b7-a201-6a488c7f34b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:45.435294 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a4a557-4b7d-44b7-a201-6a488c7f34b1 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:45.444765 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 509/1022] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:45 2022] GET /identity/v3/services?name=image => generated 122 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:45.445865 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d57dd-2981-4b8f-b478-dee40b3253e6 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:45.546431 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d57dd-2981-4b8f-b478-dee40b3253e6 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:45.580158 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7af840-28b6-47d1-a8e2-2fb065c6b213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:45.580810 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7af840-28b6-47d1-a8e2-2fb065c6b213 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:45.581089 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7af840-28b6-47d1-a8e2-2fb065c6b213 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:45.581356 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7af840-28b6-47d1-a8e2-2fb065c6b213 None admin] PATH_INFO: `/v3/services` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:45.581974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7af840-28b6-47d1-a8e2-2fb065c6b213 None admin] RBAC: Adding query filter params (type=image) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:45.582272 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7af840-28b6-47d1-a8e2-2fb065c6b213 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:45.583656 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7af840-28b6-47d1-a8e2-2fb065c6b213 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:45.593992 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 514/1023] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:45 2022] GET /identity/v3/services?type=image => generated 350 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:45.595048 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a4a557-4b7d-44b7-a201-6a488c7f34b1 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:45.605016 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7af840-28b6-47d1-a8e2-2fb065c6b213 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:45.651983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:45.700133 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a4a557-4b7d-44b7-a201-6a488c7f34b1 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:45.713415 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7af840-28b6-47d1-a8e2-2fb065c6b213 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:45.734759 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d35a0-eef1-42ab-bda9-691074b2ff6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:45.735397 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d35a0-eef1-42ab-bda9-691074b2ff6e None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:45.735671 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d35a0-eef1-42ab-bda9-691074b2ff6e None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:45.735940 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d35a0-eef1-42ab-bda9-691074b2ff6e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:45.736630 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07d35a0-eef1-42ab-bda9-691074b2ff6e None admin] RBAC: Adding query filter params (interface=public, service_id=5703b632a5fe4d77878e89f1f921fa90, region_id=RegionOne) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:45.736931 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07d35a0-eef1-42ab-bda9-691074b2ff6e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:45.738295 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07d35a0-eef1-42ab-bda9-691074b2ff6e None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:45.748258 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 510/1024] 10.222.0.30 () {56 vars in 1251 bytes} [Tue Aug 30 14:30:45 2022] GET /identity/v3/endpoints?service_id=5703b632a5fe4d77878e89f1f921fa90&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:45.749348 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d35a0-eef1-42ab-bda9-691074b2ff6e None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:45.751514 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f392a214-c30a-45c2-ab62-18ed17a0f4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:45.752145 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f392a214-c30a-45c2-ab62-18ed17a0f4ea None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:45.752423 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f392a214-c30a-45c2-ab62-18ed17a0f4ea None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:45.752697 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f392a214-c30a-45c2-ab62-18ed17a0f4ea None admin] PATH_INFO: `/v3/domains/default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:45.753864 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f392a214-c30a-45c2-ab62-18ed17a0f4ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:45.755306 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f392a214-c30a-45c2-ab62-18ed17a0f4ea None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:45.756985 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 515/1025] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:30:45.762810 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f392a214-c30a-45c2-ab62-18ed17a0f4ea None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:45.797484 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:45.860960 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d35a0-eef1-42ab-bda9-691074b2ff6e None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:45.869197 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f392a214-c30a-45c2-ab62-18ed17a0f4ea None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:45.880492 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:45.896137 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941ebce5-9a46-4970-875e-ee8e9fcdee73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:45.896750 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ebce5-9a46-4970-875e-ee8e9fcdee73 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:45.897029 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ebce5-9a46-4970-875e-ee8e9fcdee73 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:45.897300 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ebce5-9a46-4970-875e-ee8e9fcdee73 None admin] PATH_INFO: `/v3/services` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:45.897895 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941ebce5-9a46-4970-875e-ee8e9fcdee73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:45.899279 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941ebce5-9a46-4970-875e-ee8e9fcdee73 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:45.906585 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 511/1026] 10.222.0.30 () {56 vars in 1087 bytes} [Tue Aug 30 14:30:45 2022] GET /identity/v3/services => generated 1957 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:30:45.907873 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ce4d3-78c5-4ba1-9e5c-349df4f72485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:45.908486 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ce4d3-78c5-4ba1-9e5c-349df4f72485 None admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:45.908767 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ce4d3-78c5-4ba1-9e5c-349df4f72485 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:45.909040 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ce4d3-78c5-4ba1-9e5c-349df4f72485 None admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:45.909706 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783ce4d3-78c5-4ba1-9e5c-349df4f72485 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:45.911156 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783ce4d3-78c5-4ba1-9e5c-349df4f72485 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:45.942802 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 516/1027] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Aug 30 14:30:45 2022] POST /identity/v3/projects => generated 311 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:30:46.782832 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab64cbb-b2ab-480e-99bf-3194e204ab9d None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:46.783391 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab64cbb-b2ab-480e-99bf-3194e204ab9d None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:46.783391 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab64cbb-b2ab-480e-99bf-3194e204ab9d None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:46.784782 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 512/1028] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:46.789923 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d4f02-1ef5-4a3d-9e7c-762c74817ddd None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:46.790351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d4f02-1ef5-4a3d-9e7c-762c74817ddd None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:46.791086 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d4f02-1ef5-4a3d-9e7c-762c74817ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:46.808887 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-ab7d4f02-1ef5-4a3d-9e7c-762c74817ddd None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:46.810311 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-ab7d4f02-1ef5-4a3d-9e7c-762c74817ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:46.879415 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 517/1029] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:46 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:30:47.154756 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab64cbb-b2ab-480e-99bf-3194e204ab9d None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:47.266439 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab64cbb-b2ab-480e-99bf-3194e204ab9d None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:47.273507 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:47.303236 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cf7883-0de5-400b-b28b-7bdf3dc766a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:47.303895 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf7883-0de5-400b-b28b-7bdf3dc766a2 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:47.304173 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf7883-0de5-400b-b28b-7bdf3dc766a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:47.304443 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf7883-0de5-400b-b28b-7bdf3dc766a2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:47.315451 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cf7883-0de5-400b-b28b-7bdf3dc766a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:47.320186 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-16cf7883-0de5-400b-b28b-7bdf3dc766a2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 14:30:47.320186 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:47.320186 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:47.320186 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:47.320186 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:47.320186 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:47.320186 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:47.320186 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:47.320186 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:47.320186 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:47.320186 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:47.320186 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:47.320186 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:47.320186 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:47.320186 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:47.320186 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:47.321139 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:47.321139 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:47.321139 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:47.321139 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:47.321139 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:47.321139 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:47.321139 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:47.321139 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:47.321139 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:47.321139 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:47.321139 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:47.321139 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:47.321139 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:47.321139 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:47.321139 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:47.321139 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:47.322008 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:47.322008 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:47.322008 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:47.322008 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:47.322008 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:47.322008 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:47.322008 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:47.322008 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:47.322008 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:47.322008 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:47.322008 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 14:30:47.322008 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:47.323563 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 513/1030] 10.222.0.30 () {56 vars in 1120 bytes} [Tue Aug 30 14:30:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:47.327600 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d4f02-1ef5-4a3d-9e7c-762c74817ddd None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:47.421604 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:47.432581 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d4f02-1ef5-4a3d-9e7c-762c74817ddd None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:47.467958 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc86bd4-286b-4730-bd1b-453ead0762ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:47.468570 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc86bd4-286b-4730-bd1b-453ead0762ba None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:47.468852 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc86bd4-286b-4730-bd1b-453ead0762ba None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:47.469126 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc86bd4-286b-4730-bd1b-453ead0762ba None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:47.469795 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc86bd4-286b-4730-bd1b-453ead0762ba None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:47.470124 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc86bd4-286b-4730-bd1b-453ead0762ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:47.471499 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc86bd4-286b-4730-bd1b-453ead0762ba None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:47.480525 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 518/1031] 10.222.0.30 () {56 vars in 1115 bytes} [Tue Aug 30 14:30:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:47.486048 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cf7883-0de5-400b-b28b-7bdf3dc766a2 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:47.588417 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cf7883-0de5-400b-b28b-7bdf3dc766a2 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:47.623638 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b43f5-dfd9-470e-aca1-223da1a63600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:47.624265 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b43f5-dfd9-470e-aca1-223da1a63600 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:47.624536 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b43f5-dfd9-470e-aca1-223da1a63600 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:47.624796 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b43f5-dfd9-470e-aca1-223da1a63600 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:47.635063 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8b43f5-dfd9-470e-aca1-223da1a63600 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:47.636535 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8b43f5-dfd9-470e-aca1-223da1a63600 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:47.651927 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-2d8b43f5-dfd9-470e-aca1-223da1a63600 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:47.651927 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:47.651927 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:47.651927 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:47.651927 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:47.651927 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:47.651927 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:47.651927 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:47.651927 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:47.651927 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:47.651927 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:47.651927 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:47.651927 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:30:47.651927 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:30:47.651927 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:30:47.651927 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:30:47.652912 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:47.652912 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:47.652912 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:47.652912 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:47.652912 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:47.652912 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:47.652912 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:47.652912 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:47.652912 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:47.652912 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:47.652912 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:47.652912 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:47.652912 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:47.652912 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:47.652912 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:47.652912 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:47.652912 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:30:47.653817 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:30:47.653817 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:47.653817 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:47.655535 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 514/1032] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:47.658992 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc86bd4-286b-4730-bd1b-453ead0762ba None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:47.765547 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc86bd4-286b-4730-bd1b-453ead0762ba None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:47.798946 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920e0c5c-d315-4107-be31-df8b711384fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:47.799623 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e0c5c-d315-4107-be31-df8b711384fa None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:47.799897 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e0c5c-d315-4107-be31-df8b711384fa None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:47.800161 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e0c5c-d315-4107-be31-df8b711384fa None admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:47.800794 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920e0c5c-d315-4107-be31-df8b711384fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:47.801090 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920e0c5c-d315-4107-be31-df8b711384fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:47.802436 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920e0c5c-d315-4107-be31-df8b711384fa None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:47.815659 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 519/1033] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:30:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:47.821138 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b43f5-dfd9-470e-aca1-223da1a63600 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:47.843934 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:47.849599 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677cbe4-58f1-4ab8-9ece-3f083b09f42a None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:47.849887 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677cbe4-58f1-4ab8-9ece-3f083b09f42a None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:47.850163 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677cbe4-58f1-4ab8-9ece-3f083b09f42a None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:47.851674 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 520/1034] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:47.856915 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4164b66-1875-4f43-9732-37dfdc4d5b70 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:47.857202 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4164b66-1875-4f43-9732-37dfdc4d5b70 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:47.857478 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4164b66-1875-4f43-9732-37dfdc4d5b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:47.876167 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-f4164b66-1875-4f43-9732-37dfdc4d5b70 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:47.877367 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-f4164b66-1875-4f43-9732-37dfdc4d5b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:47.887939 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:47.938296 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b43f5-dfd9-470e-aca1-223da1a63600 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:47.950508 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 521/1035] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:47 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:30:47.951955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721bc8b-35ec-4674-900c-68d22a985656 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:47.952249 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721bc8b-35ec-4674-900c-68d22a985656 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:47.952524 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721bc8b-35ec-4674-900c-68d22a985656 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:47.953922 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 522/1036] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:47.959125 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b502e6-8483-4014-83f5-3af64171b6ee None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:47.959442 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b502e6-8483-4014-83f5-3af64171b6ee None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:47.959719 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b502e6-8483-4014-83f5-3af64171b6ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:47.978208 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c394e37-9086-4463-9f2e-fb15d6398259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:47.978556 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-91b502e6-8483-4014-83f5-3af64171b6ee None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:47.979316 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c394e37-9086-4463-9f2e-fb15d6398259 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:47.979781 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-91b502e6-8483-4014-83f5-3af64171b6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:47.979918 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c394e37-9086-4463-9f2e-fb15d6398259 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:47.980335 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c394e37-9086-4463-9f2e-fb15d6398259 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:47.988345 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c394e37-9086-4463-9f2e-fb15d6398259 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:47.989990 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c394e37-9086-4463-9f2e-fb15d6398259 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:48.010862 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-5c394e37-9086-4463-9f2e-fb15d6398259 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:30:48.010862 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:48.010862 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:48.010862 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:48.010862 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:48.010862 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:48.010862 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:48.010862 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:48.010862 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:48.010862 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:48.010862 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:48.010862 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:48.010862 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:30:48.010862 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:30:48.010862 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:30:48.010862 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:30:48.011969 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:48.011969 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:48.011969 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:30:48.011969 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:30:48.011969 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:30:48.011969 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:30:48.011969 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:48.011969 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:48.011969 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:48.011969 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:48.011969 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:48.011969 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:48.011969 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:48.011969 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:48.011969 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:48.011969 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:48.011969 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:48.011969 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:48.012939 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:48.012939 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:48.012939 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:30:48.012939 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:30:48.012939 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:30:48.012939 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:30:48.012939 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:30:48.012939 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:30:48.012939 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:30:48.012939 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:48.012939 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 515/1037] 10.222.0.30 () {56 vars in 1093 bytes} [Tue Aug 30 14:30:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:48.021706 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c394e37-9086-4463-9f2e-fb15d6398259 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:48.057765 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 523/1038] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:47 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:30:48.107574 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:48.128525 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c394e37-9086-4463-9f2e-fb15d6398259 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:48.161296 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36774fd9-d765-4312-bebc-9bc2fe0ceb69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:48.161902 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36774fd9-d765-4312-bebc-9bc2fe0ceb69 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:48.162183 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36774fd9-d765-4312-bebc-9bc2fe0ceb69 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:48.162455 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36774fd9-d765-4312-bebc-9bc2fe0ceb69 None admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:48.163177 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36774fd9-d765-4312-bebc-9bc2fe0ceb69 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:48.163508 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36774fd9-d765-4312-bebc-9bc2fe0ceb69 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:48.164965 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36774fd9-d765-4312-bebc-9bc2fe0ceb69 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:48.182497 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 516/1039] 10.222.0.30 () {56 vars in 1133 bytes} [Tue Aug 30 14:30:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:48.188331 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b502e6-8483-4014-83f5-3af64171b6ee None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:48.222028 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36774fd9-d765-4312-bebc-9bc2fe0ceb69 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:48.276051 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:48.302660 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b502e6-8483-4014-83f5-3af64171b6ee None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:48.328342 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36774fd9-d765-4312-bebc-9bc2fe0ceb69 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:48.338835 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ca94a2-32f6-4703-ad71-863228f0c57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:48.339476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca94a2-32f6-4703-ad71-863228f0c57c None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:48.339753 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca94a2-32f6-4703-ad71-863228f0c57c None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:48.340021 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca94a2-32f6-4703-ad71-863228f0c57c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:48.363621 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f82381e-539e-463b-8d4e-577e826a6ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:48.364248 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f82381e-539e-463b-8d4e-577e826a6ccf None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:48.364530 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f82381e-539e-463b-8d4e-577e826a6ccf None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:48.364809 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f82381e-539e-463b-8d4e-577e826a6ccf None admin] PATH_INFO: `/v3/services/image` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:48.365386 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f82381e-539e-463b-8d4e-577e826a6ccf None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:48.367065 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f82381e-539e-463b-8d4e-577e826a6ccf None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:48.370529 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:48.377054 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-0f82381e-539e-463b-8d4e-577e826a6ccf None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 14:30:48.377054 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:48.377054 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:30:48.377054 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:30:48.377054 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:30:48.377054 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:30:48.377054 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:30:48.377054 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:30:48.377054 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 14:30:48.377054 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:48.377054 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:30:48.377054 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:48.377054 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:48.377054 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:48.377054 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:48.377054 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:48.377054 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:48.378167 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:48.378167 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:48.378167 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:48.378167 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:48.378167 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:48.378167 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:48.378167 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:30:48.378167 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:30:48.378167 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:30:48.378167 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:30:48.378167 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:48.378167 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:48.378167 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:48.378167 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:48.378167 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:48.378167 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:48.379134 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:48.379134 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:48.379134 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:48.379134 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:48.379134 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:48.379134 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:48.379134 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:48.379134 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:48.379134 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:48.379134 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:48.379134 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:30:48.379134 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:30:48.379134 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 14:30:48.379134 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:48.379134 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 517/1040] 10.222.0.30 () {56 vars in 1105 bytes} [Tue Aug 30 14:30:48 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:30:48.380636 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f82381e-539e-463b-8d4e-577e826a6ccf None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:48.419104 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ca94a2-32f6-4703-ad71-863228f0c57c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:48.420598 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ca94a2-32f6-4703-ad71-863228f0c57c None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:48.477561 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f82381e-539e-463b-8d4e-577e826a6ccf None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:48.495366 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:48.511934 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d65b02-0fa4-47aa-87d9-649137a78b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:48.512322 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d65b02-0fa4-47aa-87d9-649137a78b4b None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:48.512606 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d65b02-0fa4-47aa-87d9-649137a78b4b None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:48.512867 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d65b02-0fa4-47aa-87d9-649137a78b4b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:48.513977 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d65b02-0fa4-47aa-87d9-649137a78b4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:48.515425 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d65b02-0fa4-47aa-87d9-649137a78b4b None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:48.516996 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 518/1041] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:48.517746 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d65b02-0fa4-47aa-87d9-649137a78b4b None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:48.528482 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 524/1042] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:30:48 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:30:48.528482 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ca94a2-32f6-4703-ad71-863228f0c57c None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:48.584643 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:48.619586 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d65b02-0fa4-47aa-87d9-649137a78b4b None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:48.631454 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ca94a2-32f6-4703-ad71-863228f0c57c None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:48.652359 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43104c54-83cc-46c2-ab88-770fc137a44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:48.652962 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43104c54-83cc-46c2-ab88-770fc137a44d None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:48.653244 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43104c54-83cc-46c2-ab88-770fc137a44d None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:48.653511 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43104c54-83cc-46c2-ab88-770fc137a44d None admin] PATH_INFO: `/v3/services` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:48.654127 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43104c54-83cc-46c2-ab88-770fc137a44d None admin] RBAC: Adding query filter params (name=image) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:48.654422 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43104c54-83cc-46c2-ab88-770fc137a44d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:48.655813 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43104c54-83cc-46c2-ab88-770fc137a44d None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:48.660822 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 519/1043] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:48 2022] GET /identity/v3/services?name=image => generated 122 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:48.661588 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43104c54-83cc-46c2-ab88-770fc137a44d None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:48.669202 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b90ef8-f173-4d84-8b0f-3c54fb3bb646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:48.669814 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b90ef8-f173-4d84-8b0f-3c54fb3bb646 None admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:48.670090 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b90ef8-f173-4d84-8b0f-3c54fb3bb646 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:48.670360 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b90ef8-f173-4d84-8b0f-3c54fb3bb646 None admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:48.671013 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b90ef8-f173-4d84-8b0f-3c54fb3bb646 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:48.672459 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b90ef8-f173-4d84-8b0f-3c54fb3bb646 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:48.699896 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 525/1044] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 30 14:30:48 2022] POST /identity/v3/users => generated 285 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:30:48.699896 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b90ef8-f173-4d84-8b0f-3c54fb3bb646 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:48.737880 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:48.785862 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43104c54-83cc-46c2-ab88-770fc137a44d None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:48.820394 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b90ef8-f173-4d84-8b0f-3c54fb3bb646 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:48.820987 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:48.827502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d1221b-bbd7-4f5a-b956-a96fb86188fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:48.828882 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d1221b-bbd7-4f5a-b956-a96fb86188fe None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:48.829311 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d1221b-bbd7-4f5a-b956-a96fb86188fe None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:48.829772 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d1221b-bbd7-4f5a-b956-a96fb86188fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:48.842972 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d1221b-bbd7-4f5a-b956-a96fb86188fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:48.845087 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d1221b-bbd7-4f5a-b956-a96fb86188fe None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:48.861302 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073a8926-1296-4dc6-aa4f-5b77467178f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:48.861594 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-37d1221b-bbd7-4f5a-b956-a96fb86188fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:48.861594 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:48.861594 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:48.861594 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:48.861594 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:48.861594 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:48.861594 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:48.861594 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:48.861594 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:48.861594 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:48.861594 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:48.861594 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:48.861594 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:30:48.861594 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:30:48.861594 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:30:48.861594 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:30:48.862554 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:48.862554 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:48.862554 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:48.862554 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:48.862554 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:48.862554 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:48.862554 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:48.862554 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:48.862554 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:48.862554 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:48.862554 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:48.862554 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:48.862554 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:48.862554 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:48.862554 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:48.862554 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:48.862554 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:30:48.864005 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:30:48.864005 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:48.864005 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:48.864198 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a8926-1296-4dc6-aa4f-5b77467178f8 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:48.864198 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a8926-1296-4dc6-aa4f-5b77467178f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:48.864341 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 520/1045] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:48.864411 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a8926-1296-4dc6-aa4f-5b77467178f8 None admin] PATH_INFO: `/v3/services` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:48.864411 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073a8926-1296-4dc6-aa4f-5b77467178f8 None admin] RBAC: Adding query filter params (type=image) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:48.864547 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073a8926-1296-4dc6-aa4f-5b77467178f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:48.865879 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073a8926-1296-4dc6-aa4f-5b77467178f8 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:48.869158 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d1221b-bbd7-4f5a-b956-a96fb86188fe None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:48.873215 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 526/1046] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:48 2022] GET /identity/v3/services?type=image => generated 350 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:48.878521 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073a8926-1296-4dc6-aa4f-5b77467178f8 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:48.979773 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d1221b-bbd7-4f5a-b956-a96fb86188fe None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:48.985863 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:48.993068 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073a8926-1296-4dc6-aa4f-5b77467178f8 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:49.020742 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93698b49-f4a3-44a7-b1ab-92f8a0382688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:49.021352 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93698b49-f4a3-44a7-b1ab-92f8a0382688 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:49.021630 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93698b49-f4a3-44a7-b1ab-92f8a0382688 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:49.022058 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93698b49-f4a3-44a7-b1ab-92f8a0382688 None admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:49.022539 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93698b49-f4a3-44a7-b1ab-92f8a0382688 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:49.022847 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93698b49-f4a3-44a7-b1ab-92f8a0382688 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:49.024245 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93698b49-f4a3-44a7-b1ab-92f8a0382688 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:49.030596 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d86b7f-9532-4d40-8936-db60eba1c40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:49.031226 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d86b7f-9532-4d40-8936-db60eba1c40d None admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:49.031544 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d86b7f-9532-4d40-8936-db60eba1c40d None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:49.031819 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d86b7f-9532-4d40-8936-db60eba1c40d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:49.032499 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d86b7f-9532-4d40-8936-db60eba1c40d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:49.033863 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d86b7f-9532-4d40-8936-db60eba1c40d None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:49.043061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 521/1047] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:30:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:30:49.045231 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93698b49-f4a3-44a7-b1ab-92f8a0382688 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:49.056143 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 527/1048] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:30:48 2022] POST /identity/v3/endpoints => generated 327 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:30:49.097693 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:49.158897 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93698b49-f4a3-44a7-b1ab-92f8a0382688 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:49.207772 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aa5952-9354-4591-b6e4-c94a8859b027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:49.207772 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa5952-9354-4591-b6e4-c94a8859b027 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:49.207772 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa5952-9354-4591-b6e4-c94a8859b027 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:49.207772 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa5952-9354-4591-b6e4-c94a8859b027 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:49.215711 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aa5952-9354-4591-b6e4-c94a8859b027 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:49.217358 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aa5952-9354-4591-b6e4-c94a8859b027 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:49.230254 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-11aa5952-9354-4591-b6e4-c94a8859b027 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:30:49.230254 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:49.230254 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:49.230254 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:49.230254 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:49.230254 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:49.230254 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:49.230254 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:49.230254 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:49.230254 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:49.230254 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:49.230254 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:49.230254 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:30:49.230254 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:30:49.230254 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:30:49.231198 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:30:49.231198 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:49.231198 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:49.231198 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:49.231198 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:49.231198 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:49.231198 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:49.231198 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:49.231198 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:49.231198 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:49.231198 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:49.231198 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:49.231198 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:49.231198 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:49.231198 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:49.231198 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:49.231198 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:49.232288 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:30:49.232288 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:30:49.232288 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:30:49.232288 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:30:49.232288 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:30:49.232288 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:30:49.232288 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:30:49.232288 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:49.232288 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 522/1049] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:30:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:30:49.237591 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d86b7f-9532-4d40-8936-db60eba1c40d None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:49.345002 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d86b7f-9532-4d40-8936-db60eba1c40d None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:49.378029 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cad3e0-5dff-447e-a6ec-81d465599ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:49.378638 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cad3e0-5dff-447e-a6ec-81d465599ff7 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:49.378918 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cad3e0-5dff-447e-a6ec-81d465599ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:49.379191 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cad3e0-5dff-447e-a6ec-81d465599ff7 None admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:49.379910 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cad3e0-5dff-447e-a6ec-81d465599ff7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:49.380214 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cad3e0-5dff-447e-a6ec-81d465599ff7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:49.381636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cad3e0-5dff-447e-a6ec-81d465599ff7 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:49.394607 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 528/1050] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 30 14:30:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:49.401011 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aa5952-9354-4591-b6e4-c94a8859b027 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:49.500497 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:49.504544 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aa5952-9354-4591-b6e4-c94a8859b027 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:49.537971 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09199def-2415-472e-9947-cff348883d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:49.540600 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09199def-2415-472e-9947-cff348883d61 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:49.540600 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09199def-2415-472e-9947-cff348883d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:49.540600 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09199def-2415-472e-9947-cff348883d61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:49.540600 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09199def-2415-472e-9947-cff348883d61 None admin] RBAC: Adding query filter params (role.id=2345dd1fbf154d6888e8113c51914966, scope.project.id=01cf71ca5abf4576b89d10c85af6d2de, user.id=8878de66a03044e38e507028853a0cd1) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:49.541024 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09199def-2415-472e-9947-cff348883d61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:49.542280 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09199def-2415-472e-9947-cff348883d61 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:49.548067 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 523/1051] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:30:49 2022] GET /identity/v3/role_assignments?user.id=8878de66a03044e38e507028853a0cd1&scope.project.id=01cf71ca5abf4576b89d10c85af6d2de&role.id=2345dd1fbf154d6888e8113c51914966 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:50.668348 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453cdecb-7fcf-4e35-9abf-1735ede45aca None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:50.668867 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453cdecb-7fcf-4e35-9abf-1735ede45aca None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:50.668867 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453cdecb-7fcf-4e35-9abf-1735ede45aca None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:50.670383 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 529/1052] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:50.675367 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc3756-618d-404d-a689-b709623d4942 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:50.675807 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc3756-618d-404d-a689-b709623d4942 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:50.675952 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc3756-618d-404d-a689-b709623d4942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:50.694473 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-26cc3756-618d-404d-a689-b709623d4942 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:50.695685 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-26cc3756-618d-404d-a689-b709623d4942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:50.771276 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 524/1053] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:30:51.001777 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f07303-c64a-4745-99dd-e1901aa3b2fc None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:51.001965 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f07303-c64a-4745-99dd-e1901aa3b2fc None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:51.002228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f07303-c64a-4745-99dd-e1901aa3b2fc None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:51.003743 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 530/1054] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:51.008698 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3879dd-bd22-44d4-9137-22e49363c75a None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:51.009525 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3879dd-bd22-44d4-9137-22e49363c75a None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:51.009823 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3879dd-bd22-44d4-9137-22e49363c75a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:51.028120 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-9f3879dd-bd22-44d4-9137-22e49363c75a None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:51.029283 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-9f3879dd-bd22-44d4-9137-22e49363c75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:51.039364 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f07303-c64a-4745-99dd-e1901aa3b2fc None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:51.051584 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:51.102393 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 525/1055] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:30:51.147775 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f07303-c64a-4745-99dd-e1901aa3b2fc None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:51.182683 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c3e415-d233-4cb7-ba4b-3aad4419ba69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:51.183583 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3e415-d233-4cb7-ba4b-3aad4419ba69 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:51.184294 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3e415-d233-4cb7-ba4b-3aad4419ba69 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:51.184732 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3e415-d233-4cb7-ba4b-3aad4419ba69 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:51.195137 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c3e415-d233-4cb7-ba4b-3aad4419ba69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:51.200266 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-f4c3e415-d233-4cb7-ba4b-3aad4419ba69 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:30:51.200266 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:51.200266 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:51.200266 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:51.200266 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:51.200266 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:51.200266 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:51.200266 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:51.200266 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:51.200266 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:51.200266 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:51.200266 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:51.200266 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:51.200266 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:51.200266 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:51.200266 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:51.201382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:51.201382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:51.201382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:51.201382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:51.201382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:51.201382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:51.201382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:51.201382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:51.201382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:51.201382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:51.201382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:51.201382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:51.201382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:51.201382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:51.201382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:51.201382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:51.202414 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:51.202414 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:51.202414 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:51.202414 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:51.202414 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:51.202414 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:51.202414 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:51.202414 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:51.202414 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:51.202414 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:51.202414 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:30:51.202414 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:51.204353 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 531/1056] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 30 14:30:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:30:51.213515 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3879dd-bd22-44d4-9137-22e49363c75a None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:51.242436 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:51.320935 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3879dd-bd22-44d4-9137-22e49363c75a None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:51.363233 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b344a76e-c2cc-457c-82a4-7e0e2ab92e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:51.363846 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344a76e-c2cc-457c-82a4-7e0e2ab92e26 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:51.364126 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344a76e-c2cc-457c-82a4-7e0e2ab92e26 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:51.364396 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344a76e-c2cc-457c-82a4-7e0e2ab92e26 None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:51.365113 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b344a76e-c2cc-457c-82a4-7e0e2ab92e26 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:51.365418 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b344a76e-c2cc-457c-82a4-7e0e2ab92e26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:51.366776 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b344a76e-c2cc-457c-82a4-7e0e2ab92e26 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:51.372802 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c3e415-d233-4cb7-ba4b-3aad4419ba69 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:51.376069 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 526/1057] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:30:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:51.381517 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b344a76e-c2cc-457c-82a4-7e0e2ab92e26 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:51.452852 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:51.472719 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:51.484913 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c3e415-d233-4cb7-ba4b-3aad4419ba69 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:51.487856 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b344a76e-c2cc-457c-82a4-7e0e2ab92e26 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:51.518525 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8353a5e9-2dc5-455b-86ae-2f67d405bc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:51.519124 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8353a5e9-2dc5-455b-86ae-2f67d405bc2e None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:51.519439 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8353a5e9-2dc5-455b-86ae-2f67d405bc2e None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:51.519712 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8353a5e9-2dc5-455b-86ae-2f67d405bc2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:51.520999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bbb323-1be2-4300-bae0-4940b969f50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:51.521622 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bbb323-1be2-4300-bae0-4940b969f50b None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:51.521896 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bbb323-1be2-4300-bae0-4940b969f50b None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:51.522165 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bbb323-1be2-4300-bae0-4940b969f50b None admin] PATH_INFO: `/v3/users/73cae25f4b724e7e908b1138f58a66f8` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:51.534995 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bbb323-1be2-4300-bae0-4940b969f50b None admin] RBAC: Authorizing `identity:get_user(user_id=73cae25f4b724e7e908b1138f58a66f8)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:51.536449 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bbb323-1be2-4300-bae0-4940b969f50b None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:51.538055 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 527/1058] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:30:51 2022] GET /identity/v3/users/73cae25f4b724e7e908b1138f58a66f8 => generated 285 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:30:51.539282 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8567bc-f3d1-42c5-bb02-b21c9b63d9bf None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:51.539572 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8567bc-f3d1-42c5-bb02-b21c9b63d9bf None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:51.539823 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8567bc-f3d1-42c5-bb02-b21c9b63d9bf None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:51.543667 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 528/1059] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:51.544427 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8567bc-f3d1-42c5-bb02-b21c9b63d9bf None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:51.590382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8353a5e9-2dc5-455b-86ae-2f67d405bc2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:51.591866 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8353a5e9-2dc5-455b-86ae-2f67d405bc2e None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:51.646210 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8567bc-f3d1-42c5-bb02-b21c9b63d9bf None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:51.658885 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:51.682433 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dae7f9-5011-40cc-81ea-6359e373438b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:51.683037 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dae7f9-5011-40cc-81ea-6359e373438b None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:51.683343 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dae7f9-5011-40cc-81ea-6359e373438b None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:51.683613 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dae7f9-5011-40cc-81ea-6359e373438b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:51.691302 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 532/1060] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:30:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 14:30:51.692687 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c7368d-2862-4e82-90a1-8d1deb7d3bf8 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:51.692976 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c7368d-2862-4e82-90a1-8d1deb7d3bf8 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:51.693248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c7368d-2862-4e82-90a1-8d1deb7d3bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:51.699642 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:51.710875 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-86c7368d-2862-4e82-90a1-8d1deb7d3bf8 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:51.712154 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-86c7368d-2862-4e82-90a1-8d1deb7d3bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:51.752661 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dae7f9-5011-40cc-81ea-6359e373438b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:51.754106 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dae7f9-5011-40cc-81ea-6359e373438b None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:51.778130 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 533/1061] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:30:51.779082 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c7368d-2862-4e82-90a1-8d1deb7d3bf8 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:51.863925 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 529/1062] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:30:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 14:30:51.869615 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dae7f9-5011-40cc-81ea-6359e373438b None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:51.887825 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:51.890763 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c7368d-2862-4e82-90a1-8d1deb7d3bf8 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:51.918926 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:51.929341 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3623b97-7ba7-4499-807a-59b926e1053e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:51.930269 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3623b97-7ba7-4499-807a-59b926e1053e None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:51.930544 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3623b97-7ba7-4499-807a-59b926e1053e None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:51.930955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3623b97-7ba7-4499-807a-59b926e1053e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:51.942054 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3623b97-7ba7-4499-807a-59b926e1053e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:51.943707 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3623b97-7ba7-4499-807a-59b926e1053e None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:51.958306 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-e3623b97-7ba7-4499-807a-59b926e1053e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:51.958306 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:51.958306 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:51.958306 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:51.958306 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:51.958306 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:51.958306 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:51.958306 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:51.958306 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:51.958306 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:51.958306 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:51.958306 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:51.958306 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:30:51.958306 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:30:51.958306 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:30:51.958306 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:30:51.959274 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:51.959274 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:51.959274 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:51.959274 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:51.959274 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:51.959274 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:51.959274 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:51.959274 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:51.959274 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:51.959274 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:51.959274 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:51.959274 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:51.959274 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:51.959274 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:51.959274 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:51.959274 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:51.959274 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:30:51.960175 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:30:51.960175 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:51.960175 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:51.963555 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 534/1063] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:51.966135 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3623b97-7ba7-4499-807a-59b926e1053e None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:51.984776 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dae7f9-5011-40cc-81ea-6359e373438b None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:52.020005 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b829de6-a679-4f84-a4d1-d1b49caf3e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:52.020636 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b829de6-a679-4f84-a4d1-d1b49caf3e53 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:52.020926 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b829de6-a679-4f84-a4d1-d1b49caf3e53 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:52.021195 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b829de6-a679-4f84-a4d1-d1b49caf3e53 None admin] PATH_INFO: `/v3/projects/ba3cb001c0304cb3bbefeded2355d23f` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:52.022328 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b829de6-a679-4f84-a4d1-d1b49caf3e53 None admin] RBAC: Authorizing `identity:get_project(project_id=ba3cb001c0304cb3bbefeded2355d23f)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:52.023794 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b829de6-a679-4f84-a4d1-d1b49caf3e53 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:52.026027 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 530/1064] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:30:51 2022] GET /identity/v3/projects/ba3cb001c0304cb3bbefeded2355d23f => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:30:52.031993 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b829de6-a679-4f84-a4d1-d1b49caf3e53 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:52.071630 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3623b97-7ba7-4499-807a-59b926e1053e None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:52.105579 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7325345f-a05f-441c-b4e3-f5f216d975b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:52.106188 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325345f-a05f-441c-b4e3-f5f216d975b8 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:52.106469 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325345f-a05f-441c-b4e3-f5f216d975b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:52.106742 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325345f-a05f-441c-b4e3-f5f216d975b8 None admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:52.107418 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7325345f-a05f-441c-b4e3-f5f216d975b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:52.107723 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7325345f-a05f-441c-b4e3-f5f216d975b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:52.109091 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7325345f-a05f-441c-b4e3-f5f216d975b8 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:52.121533 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 535/1065] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:30:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:30:52.122273 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7325345f-a05f-441c-b4e3-f5f216d975b8 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:52.135546 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b829de6-a679-4f84-a4d1-d1b49caf3e53 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:52.148594 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:52.151814 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:52.178699 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2585799d-48e1-43e2-8d76-eb4f8185f7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:52.179740 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2585799d-48e1-43e2-8d76-eb4f8185f7c8 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:52.180090 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2585799d-48e1-43e2-8d76-eb4f8185f7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:52.180371 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2585799d-48e1-43e2-8d76-eb4f8185f7c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:52.181402 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2585799d-48e1-43e2-8d76-eb4f8185f7c8 None admin] RBAC: Adding query filter params (role.id=3cda0615d87d4af0860a2a5e9b4e2c05, scope.project.id=ba3cb001c0304cb3bbefeded2355d23f, user.id=73cae25f4b724e7e908b1138f58a66f8) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:52.181707 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2585799d-48e1-43e2-8d76-eb4f8185f7c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:52.183181 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2585799d-48e1-43e2-8d76-eb4f8185f7c8 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:52.190166 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 531/1066] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:30:52 2022] GET /identity/v3/role_assignments?user.id=73cae25f4b724e7e908b1138f58a66f8&scope.project.id=ba3cb001c0304cb3bbefeded2355d23f&role.id=3cda0615d87d4af0860a2a5e9b4e2c05 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:30:52.257416 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7325345f-a05f-441c-b4e3-f5f216d975b8 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:52.303051 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da34e1c-2075-4750-944f-bf057614f2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:52.304438 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da34e1c-2075-4750-944f-bf057614f2c1 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:52.304438 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da34e1c-2075-4750-944f-bf057614f2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:52.304438 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da34e1c-2075-4750-944f-bf057614f2c1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:52.318071 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da34e1c-2075-4750-944f-bf057614f2c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:52.323487 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-0da34e1c-2075-4750-944f-bf057614f2c1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 14:30:52.323487 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:52.323487 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:52.323487 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:52.323487 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:52.323487 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:52.323487 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:52.323487 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:52.323487 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:52.323487 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:52.323487 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:52.323487 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:52.323487 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:52.323487 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:52.323487 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:52.323487 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:52.324495 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:52.324495 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:52.324495 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:52.324495 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:52.324495 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:52.324495 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:52.324495 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:52.324495 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:52.324495 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:52.324495 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:52.324495 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:52.324495 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:52.324495 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:52.324495 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:52.324495 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:52.324495 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:52.325797 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:52.325797 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:52.325797 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:52.325797 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:52.325797 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:52.325797 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:52.325797 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:52.325797 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:52.325797 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:52.325797 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:52.325797 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 14:30:52.325797 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:52.325797 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 536/1067] 10.222.0.30 () {56 vars in 1120 bytes} [Tue Aug 30 14:30:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:52.330945 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2585799d-48e1-43e2-8d76-eb4f8185f7c8 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:52.438538 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2585799d-48e1-43e2-8d76-eb4f8185f7c8 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:52.475185 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d073d421-4f6a-4c4f-936f-5a2c61fd14ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:52.475836 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d073d421-4f6a-4c4f-936f-5a2c61fd14ff None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:52.476114 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d073d421-4f6a-4c4f-936f-5a2c61fd14ff None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:52.476383 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d073d421-4f6a-4c4f-936f-5a2c61fd14ff None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:52.477038 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d073d421-4f6a-4c4f-936f-5a2c61fd14ff None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:52.477336 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d073d421-4f6a-4c4f-936f-5a2c61fd14ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:52.478765 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d073d421-4f6a-4c4f-936f-5a2c61fd14ff None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:52.486846 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 532/1068] 10.222.0.30 () {56 vars in 1115 bytes} [Tue Aug 30 14:30:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:52.492771 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da34e1c-2075-4750-944f-bf057614f2c1 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:52.563304 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:52.595761 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da34e1c-2075-4750-944f-bf057614f2c1 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:52.631064 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29db6423-1a11-4642-b33b-d4919951fa31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:52.631711 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db6423-1a11-4642-b33b-d4919951fa31 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:52.631988 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db6423-1a11-4642-b33b-d4919951fa31 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:52.632257 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db6423-1a11-4642-b33b-d4919951fa31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:52.643354 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29db6423-1a11-4642-b33b-d4919951fa31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:52.644789 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29db6423-1a11-4642-b33b-d4919951fa31 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:52.659537 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-29db6423-1a11-4642-b33b-d4919951fa31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:52.659537 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:52.659537 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:52.659537 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:52.659537 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:52.659537 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:52.659537 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:52.659537 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:52.659537 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:52.659537 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:52.659537 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:52.659537 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:52.659537 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:30:52.659537 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:30:52.659537 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:30:52.659537 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:30:52.660517 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:52.660517 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:52.660517 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:52.660517 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:52.660517 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:52.660517 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:52.660517 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:52.660517 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:52.660517 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:52.660517 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:52.660517 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:52.660517 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:52.660517 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:52.660517 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:52.660517 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:52.660517 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:52.660517 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:30:52.661415 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:30:52.661415 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:52.661415 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:52.661415 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 537/1069] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:52.666917 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d073d421-4f6a-4c4f-936f-5a2c61fd14ff None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:52.707261 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:52.769611 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d073d421-4f6a-4c4f-936f-5a2c61fd14ff None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:52.803545 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac50c7b0-cc13-402d-a8ad-7f6c0d97df93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:52.804150 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac50c7b0-cc13-402d-a8ad-7f6c0d97df93 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:52.804424 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac50c7b0-cc13-402d-a8ad-7f6c0d97df93 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:52.804690 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac50c7b0-cc13-402d-a8ad-7f6c0d97df93 None admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:52.805322 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac50c7b0-cc13-402d-a8ad-7f6c0d97df93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:52.806323 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac50c7b0-cc13-402d-a8ad-7f6c0d97df93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:52.808079 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac50c7b0-cc13-402d-a8ad-7f6c0d97df93 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:52.819960 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 533/1070] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:30:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:52.826315 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29db6423-1a11-4642-b33b-d4919951fa31 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:52.925551 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29db6423-1a11-4642-b33b-d4919951fa31 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:52.948493 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:52.959313 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f3422f-72d2-4736-a0b9-902939ea71b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:52.959903 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3422f-72d2-4736-a0b9-902939ea71b1 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:52.960180 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3422f-72d2-4736-a0b9-902939ea71b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:52.960450 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3422f-72d2-4736-a0b9-902939ea71b1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:52.968368 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f3422f-72d2-4736-a0b9-902939ea71b1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:52.969799 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f3422f-72d2-4736-a0b9-902939ea71b1 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:52.980426 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-72f3422f-72d2-4736-a0b9-902939ea71b1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:30:52.980426 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:52.980426 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:52.980426 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:52.980426 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:52.980426 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:52.980426 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:52.980426 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:52.980426 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:52.980426 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:52.980426 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:52.980426 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:52.980426 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:30:52.980426 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:30:52.980426 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:30:52.980426 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:30:52.981422 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:52.981422 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:52.981422 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:30:52.981422 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:30:52.981422 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:30:52.981422 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:30:52.981422 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:52.981422 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:52.981422 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:52.981422 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:52.981422 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:52.981422 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:52.981422 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:52.981422 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:52.981422 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:52.981422 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:52.981422 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:52.981422 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:52.982743 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:52.982743 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:52.982743 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:30:52.982743 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:30:52.982743 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:30:52.982743 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:30:52.982743 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:30:52.982743 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:30:52.982743 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:30:52.982743 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:52.982743 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 538/1071] 10.222.0.30 () {56 vars in 1093 bytes} [Tue Aug 30 14:30:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:30:52.987306 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac50c7b0-cc13-402d-a8ad-7f6c0d97df93 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:53.081097 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:53.084798 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac50c7b0-cc13-402d-a8ad-7f6c0d97df93 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:53.118598 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31db018-f828-46fe-8ab3-a7d59cae88ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:53.119195 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31db018-f828-46fe-8ab3-a7d59cae88ec None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:53.119502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31db018-f828-46fe-8ab3-a7d59cae88ec None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:53.119772 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31db018-f828-46fe-8ab3-a7d59cae88ec None admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:53.120490 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31db018-f828-46fe-8ab3-a7d59cae88ec None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:53.120791 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31db018-f828-46fe-8ab3-a7d59cae88ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:53.122205 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31db018-f828-46fe-8ab3-a7d59cae88ec None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:53.139514 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 534/1072] 10.222.0.30 () {56 vars in 1133 bytes} [Tue Aug 30 14:30:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:53.145216 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f3422f-72d2-4736-a0b9-902939ea71b1 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:53.245522 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f3422f-72d2-4736-a0b9-902939ea71b1 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:53.281181 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502255ab-d91b-405e-8cab-59c616cea2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:53.281818 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502255ab-d91b-405e-8cab-59c616cea2a9 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:53.282101 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502255ab-d91b-405e-8cab-59c616cea2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:53.282376 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502255ab-d91b-405e-8cab-59c616cea2a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:53.293808 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502255ab-d91b-405e-8cab-59c616cea2a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:53.295312 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502255ab-d91b-405e-8cab-59c616cea2a9 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:53.309626 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-502255ab-d91b-405e-8cab-59c616cea2a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:53.309626 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:53.309626 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:53.309626 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:53.309626 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:53.309626 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:53.309626 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:53.309626 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:53.309626 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:53.309626 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:53.309626 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:53.309626 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:53.309626 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:30:53.309626 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:30:53.309626 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:30:53.309626 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:30:53.310735 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:53.310735 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:53.310735 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:53.310735 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:53.310735 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:53.310735 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:53.310735 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:53.310735 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:53.310735 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:53.310735 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:53.310735 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:53.310735 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:53.310735 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:53.310735 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:53.310735 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:53.310735 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:53.310735 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:30:53.311783 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:30:53.311783 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:53.311783 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:53.311783 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 539/1073] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:53.316558 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31db018-f828-46fe-8ab3-a7d59cae88ec None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:53.414182 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31db018-f828-46fe-8ab3-a7d59cae88ec None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:53.447912 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58980032-49a7-4be9-ac68-02d977814428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:53.448508 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58980032-49a7-4be9-ac68-02d977814428 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:53.448792 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58980032-49a7-4be9-ac68-02d977814428 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:53.449062 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58980032-49a7-4be9-ac68-02d977814428 None admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:53.449692 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58980032-49a7-4be9-ac68-02d977814428 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:53.449991 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58980032-49a7-4be9-ac68-02d977814428 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:53.451376 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58980032-49a7-4be9-ac68-02d977814428 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:53.462420 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 535/1074] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:30:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:53.467906 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502255ab-d91b-405e-8cab-59c616cea2a9 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:53.502648 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:53.567495 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502255ab-d91b-405e-8cab-59c616cea2a9 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:53.600779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fbca21-9b20-45ce-b266-d2b0b6bccce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:53.601408 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fbca21-9b20-45ce-b266-d2b0b6bccce2 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:53.601694 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fbca21-9b20-45ce-b266-d2b0b6bccce2 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:53.601968 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fbca21-9b20-45ce-b266-d2b0b6bccce2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:53.608964 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fbca21-9b20-45ce-b266-d2b0b6bccce2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:53.610402 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fbca21-9b20-45ce-b266-d2b0b6bccce2 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:53.622070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-d3fbca21-9b20-45ce-b266-d2b0b6bccce2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:30:53.622070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:53.622070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:53.622070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:53.622070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:53.622070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:53.622070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:53.622070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:53.622070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:53.622070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:53.622070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:53.622070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:53.622070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:30:53.622070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:30:53.622070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:30:53.622952 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:30:53.622952 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:53.622952 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:53.622952 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:53.622952 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:53.622952 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:53.622952 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:53.622952 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:53.622952 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:53.622952 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:53.622952 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:53.622952 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:53.622952 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:53.622952 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:53.622952 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:53.622952 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:53.622952 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:53.623885 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:30:53.623885 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:30:53.623885 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:30:53.623885 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:30:53.623885 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:30:53.623885 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:30:53.623885 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:30:53.623885 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:53.623885 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 540/1075] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:30:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:53.628934 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58980032-49a7-4be9-ac68-02d977814428 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:53.648101 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:53.729886 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58980032-49a7-4be9-ac68-02d977814428 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:53.763287 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b798f5-76c6-4c0a-bca3-0542f0836cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:53.763897 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b798f5-76c6-4c0a-bca3-0542f0836cbc None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:53.764177 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b798f5-76c6-4c0a-bca3-0542f0836cbc None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:53.764453 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b798f5-76c6-4c0a-bca3-0542f0836cbc None admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:53.765139 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b798f5-76c6-4c0a-bca3-0542f0836cbc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:53.765444 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b798f5-76c6-4c0a-bca3-0542f0836cbc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:53.766873 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b798f5-76c6-4c0a-bca3-0542f0836cbc None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:53.778714 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 536/1076] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 30 14:30:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:53.784655 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fbca21-9b20-45ce-b266-d2b0b6bccce2 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:53.879449 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:53.886269 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fbca21-9b20-45ce-b266-d2b0b6bccce2 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:53.919315 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc21334-7c8e-49c0-8b7e-72108dc96589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:53.919974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc21334-7c8e-49c0-8b7e-72108dc96589 None admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:53.920252 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc21334-7c8e-49c0-8b7e-72108dc96589 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:53.920525 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc21334-7c8e-49c0-8b7e-72108dc96589 None admin] PATH_INFO: `/v3/projects/01cf71ca5abf4576b89d10c85af6d2de/users/8878de66a03044e38e507028853a0cd1/roles/2345dd1fbf154d6888e8113c51914966` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:53.922629 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc21334-7c8e-49c0-8b7e-72108dc96589 None admin] RBAC: Authorizing `identity:create_grant(project_id=01cf71ca5abf4576b89d10c85af6d2de, user_id=8878de66a03044e38e507028853a0cd1, role_id=2345dd1fbf154d6888e8113c51914966)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:53.924078 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc21334-7c8e-49c0-8b7e-72108dc96589 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:53.938346 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 541/1077] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 30 14:30:53 2022] PUT /identity/v3/projects/01cf71ca5abf4576b89d10c85af6d2de/users/8878de66a03044e38e507028853a0cd1/roles/2345dd1fbf154d6888e8113c51914966 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:30:54.111036 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a059d8a-c35e-4749-b59e-ea4858e825c5 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:54.111450 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a059d8a-c35e-4749-b59e-ea4858e825c5 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:54.111725 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a059d8a-c35e-4749-b59e-ea4858e825c5 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:54.113071 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 537/1078] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:54.118479 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44920a2a-8278-491b-b48d-554370dcb3cd None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:54.118991 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44920a2a-8278-491b-b48d-554370dcb3cd None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:54.119356 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44920a2a-8278-491b-b48d-554370dcb3cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:54.137522 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-44920a2a-8278-491b-b48d-554370dcb3cd None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:54.138695 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-44920a2a-8278-491b-b48d-554370dcb3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:54.205069 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 542/1079] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:30:54.485480 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a059d8a-c35e-4749-b59e-ea4858e825c5 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:54.562284 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:54.587664 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a059d8a-c35e-4749-b59e-ea4858e825c5 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:54.621616 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc4575e-0516-4cc4-9503-9b7be81a0ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:54.622242 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc4575e-0516-4cc4-9503-9b7be81a0ee9 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:54.622517 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc4575e-0516-4cc4-9503-9b7be81a0ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:54.622787 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc4575e-0516-4cc4-9503-9b7be81a0ee9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:54.633033 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc4575e-0516-4cc4-9503-9b7be81a0ee9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:54.637985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-4dc4575e-0516-4cc4-9503-9b7be81a0ee9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:30:54.637985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:54.637985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:54.637985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:54.637985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:54.637985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:54.637985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:54.637985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:54.637985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:54.637985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:54.637985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:54.637985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:54.637985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:54.637985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:54.637985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:54.637985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:54.639061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:54.639061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:54.639061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:54.639061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:54.639061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:54.639061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:54.639061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:54.639061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:54.639061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:54.639061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:54.639061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:54.639061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:54.639061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:54.639061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:54.639061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:54.639061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:54.639969 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:54.639969 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:54.639969 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:54.639969 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:54.639969 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:54.639969 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:54.639969 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:54.639969 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:54.639969 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:54.639969 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:54.639969 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:30:54.639969 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:54.639969 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 538/1080] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 30 14:30:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:54.645200 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44920a2a-8278-491b-b48d-554370dcb3cd None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:54.727138 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:54.742812 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44920a2a-8278-491b-b48d-554370dcb3cd None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:54.776174 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3281a15-b116-4fd3-b595-88b534e1e304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:54.776774 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3281a15-b116-4fd3-b595-88b534e1e304 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:54.777053 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3281a15-b116-4fd3-b595-88b534e1e304 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:54.777321 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3281a15-b116-4fd3-b595-88b534e1e304 None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:54.777974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3281a15-b116-4fd3-b595-88b534e1e304 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:54.778269 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3281a15-b116-4fd3-b595-88b534e1e304 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:54.779672 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3281a15-b116-4fd3-b595-88b534e1e304 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:54.787702 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 543/1081] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:30:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:54.794329 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc4575e-0516-4cc4-9503-9b7be81a0ee9 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:54.892509 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc4575e-0516-4cc4-9503-9b7be81a0ee9 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:54.927193 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc3fd42-7cd8-48c8-a100-406c3abd8ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:54.927993 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3fd42-7cd8-48c8-a100-406c3abd8ae4 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:54.928273 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3fd42-7cd8-48c8-a100-406c3abd8ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:54.928541 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3fd42-7cd8-48c8-a100-406c3abd8ae4 None admin] PATH_INFO: `/v3/users/73cae25f4b724e7e908b1138f58a66f8` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:54.929659 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc3fd42-7cd8-48c8-a100-406c3abd8ae4 None admin] RBAC: Authorizing `identity:get_user(user_id=73cae25f4b724e7e908b1138f58a66f8)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:54.931176 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc3fd42-7cd8-48c8-a100-406c3abd8ae4 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:54.932795 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 539/1082] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:30:54 2022] GET /identity/v3/users/73cae25f4b724e7e908b1138f58a66f8 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:54.938813 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3281a15-b116-4fd3-b595-88b534e1e304 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:55.039135 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3281a15-b116-4fd3-b595-88b534e1e304 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:55.073100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eefa095-f51e-4dc3-abe5-b85d10d25306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:55.073717 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eefa095-f51e-4dc3-abe5-b85d10d25306 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:55.073998 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eefa095-f51e-4dc3-abe5-b85d10d25306 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:55.074264 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eefa095-f51e-4dc3-abe5-b85d10d25306 None admin] PATH_INFO: `/v3/projects/ba3cb001c0304cb3bbefeded2355d23f` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:55.075396 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eefa095-f51e-4dc3-abe5-b85d10d25306 None admin] RBAC: Authorizing `identity:get_project(project_id=ba3cb001c0304cb3bbefeded2355d23f)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:55.076807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eefa095-f51e-4dc3-abe5-b85d10d25306 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:55.078426 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 544/1083] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:30:54 2022] GET /identity/v3/projects/ba3cb001c0304cb3bbefeded2355d23f => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:30:55.084274 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc3fd42-7cd8-48c8-a100-406c3abd8ae4 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:55.088127 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:55.188680 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc3fd42-7cd8-48c8-a100-406c3abd8ae4 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:55.224280 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4045a97d-9e7a-4eb4-8a17-919078fe2f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:55.224944 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045a97d-9e7a-4eb4-8a17-919078fe2f82 None admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:55.225217 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045a97d-9e7a-4eb4-8a17-919078fe2f82 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:55.225486 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045a97d-9e7a-4eb4-8a17-919078fe2f82 None admin] PATH_INFO: `/v3/projects/ba3cb001c0304cb3bbefeded2355d23f/users/73cae25f4b724e7e908b1138f58a66f8/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:55.227933 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4045a97d-9e7a-4eb4-8a17-919078fe2f82 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba3cb001c0304cb3bbefeded2355d23f, user_id=73cae25f4b724e7e908b1138f58a66f8, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:55.229419 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4045a97d-9e7a-4eb4-8a17-919078fe2f82 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:55.246277 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 540/1084] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 30 14:30:55 2022] PUT /identity/v3/projects/ba3cb001c0304cb3bbefeded2355d23f/users/73cae25f4b724e7e908b1138f58a66f8/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:30:55.902751 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d3d73-1bc3-45ab-bae5-496dc3175a6d None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:55.903031 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d3d73-1bc3-45ab-bae5-496dc3175a6d None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:55.903346 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d3d73-1bc3-45ab-bae5-496dc3175a6d None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:55.904724 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 545/1085] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:30:55.914613 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f0979-0f99-4a75-aded-e7b26cd2a191 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:55.915761 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f0979-0f99-4a75-aded-e7b26cd2a191 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:55.915845 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f0979-0f99-4a75-aded-e7b26cd2a191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:55.934643 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-214f0979-0f99-4a75-aded-e7b26cd2a191 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:55.935857 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-214f0979-0f99-4a75-aded-e7b26cd2a191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:55.997402 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:56.008472 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 541/1086] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:30:56.282985 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d3d73-1bc3-45ab-bae5-496dc3175a6d None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:56.311779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:56.412758 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d3d73-1bc3-45ab-bae5-496dc3175a6d None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:56.447126 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf403931-e4eb-408d-a53e-9b60cfd38d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:56.447776 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf403931-e4eb-408d-a53e-9b60cfd38d92 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:56.448060 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf403931-e4eb-408d-a53e-9b60cfd38d92 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:56.448334 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf403931-e4eb-408d-a53e-9b60cfd38d92 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:56.459173 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf403931-e4eb-408d-a53e-9b60cfd38d92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:56.464588 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-cf403931-e4eb-408d-a53e-9b60cfd38d92 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 14:30:56.464588 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:56.464588 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:56.464588 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:56.464588 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:56.464588 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:56.464588 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:56.464588 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:56.464588 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:56.464588 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:56.464588 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:56.464588 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:56.464588 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:56.464588 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:56.464588 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:56.464588 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:56.465510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:56.465510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:56.465510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:56.465510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:56.465510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:56.465510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:56.465510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:56.465510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:56.465510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:56.465510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:56.465510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:56.465510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:56.465510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:56.465510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:56.465510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:56.465510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:56.466572 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:56.466572 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:56.466572 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:56.466572 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:56.466572 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:56.466572 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:56.466572 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:56.466572 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:56.466572 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:56.466572 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:56.466572 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 14:30:56.466572 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:56.466572 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 546/1087] 10.222.0.30 () {56 vars in 1120 bytes} [Tue Aug 30 14:30:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:30:56.471620 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214f0979-0f99-4a75-aded-e7b26cd2a191 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:56.572338 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214f0979-0f99-4a75-aded-e7b26cd2a191 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:56.605536 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08525e01-b7f3-4a35-93d8-47ac10d5f49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:56.606142 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08525e01-b7f3-4a35-93d8-47ac10d5f49b None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:56.606423 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08525e01-b7f3-4a35-93d8-47ac10d5f49b None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:56.606697 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08525e01-b7f3-4a35-93d8-47ac10d5f49b None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:56.607386 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08525e01-b7f3-4a35-93d8-47ac10d5f49b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:56.607690 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08525e01-b7f3-4a35-93d8-47ac10d5f49b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:56.609047 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08525e01-b7f3-4a35-93d8-47ac10d5f49b None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:56.616899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 542/1088] 10.222.0.30 () {56 vars in 1115 bytes} [Tue Aug 30 14:30:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:56.623307 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf403931-e4eb-408d-a53e-9b60cfd38d92 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:56.700452 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:56.723452 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf403931-e4eb-408d-a53e-9b60cfd38d92 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:56.758224 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f901ed-d2f7-4585-a53c-1b0dc6ffdff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:56.758880 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f901ed-d2f7-4585-a53c-1b0dc6ffdff9 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:56.759153 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f901ed-d2f7-4585-a53c-1b0dc6ffdff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:56.759452 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f901ed-d2f7-4585-a53c-1b0dc6ffdff9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:56.769786 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f901ed-d2f7-4585-a53c-1b0dc6ffdff9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:56.771232 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f901ed-d2f7-4585-a53c-1b0dc6ffdff9 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:56.785319 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-e9f901ed-d2f7-4585-a53c-1b0dc6ffdff9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:56.785319 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:56.785319 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:56.785319 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:56.785319 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:56.785319 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:56.785319 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:56.785319 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:56.785319 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:56.785319 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:56.785319 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:56.785319 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:56.785319 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:30:56.785319 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:30:56.785319 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:30:56.785319 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:30:56.792109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:56.792109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:56.792109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:56.792109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:56.792109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:56.792109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:56.792109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:56.792109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:56.792109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:56.792109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:56.792109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:56.792109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:56.792109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:56.792109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:56.792109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:56.792109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:56.792109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:30:56.793234 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:30:56.793234 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:56.793234 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:56.793234 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 547/1089] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:30:56.793466 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08525e01-b7f3-4a35-93d8-47ac10d5f49b None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:56.848828 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:56.892925 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08525e01-b7f3-4a35-93d8-47ac10d5f49b None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:56.925899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df35f0f-74f3-4f3a-a4c8-93f2844a7726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:56.926505 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df35f0f-74f3-4f3a-a4c8-93f2844a7726 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:56.926762 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df35f0f-74f3-4f3a-a4c8-93f2844a7726 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:56.927034 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df35f0f-74f3-4f3a-a4c8-93f2844a7726 None admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:56.927701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df35f0f-74f3-4f3a-a4c8-93f2844a7726 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:56.928002 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df35f0f-74f3-4f3a-a4c8-93f2844a7726 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:56.929364 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df35f0f-74f3-4f3a-a4c8-93f2844a7726 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:56.941619 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 543/1090] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:30:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:56.947998 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f901ed-d2f7-4585-a53c-1b0dc6ffdff9 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:57.046526 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f901ed-d2f7-4585-a53c-1b0dc6ffdff9 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:57.072708 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:57.080913 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea19425-6f78-4a6b-af14-bd4a55354f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:57.081539 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea19425-6f78-4a6b-af14-bd4a55354f3d None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:57.081819 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea19425-6f78-4a6b-af14-bd4a55354f3d None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:57.082088 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea19425-6f78-4a6b-af14-bd4a55354f3d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:57.090261 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea19425-6f78-4a6b-af14-bd4a55354f3d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:57.091732 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea19425-6f78-4a6b-af14-bd4a55354f3d None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:57.102745 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-eea19425-6f78-4a6b-af14-bd4a55354f3d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:30:57.102745 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:57.102745 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:57.102745 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:57.102745 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:57.102745 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:57.102745 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:57.102745 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:57.102745 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:57.102745 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:57.102745 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:57.102745 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:57.102745 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:30:57.102745 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:30:57.102745 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:30:57.102745 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:30:57.103993 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:57.103993 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:57.103993 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:30:57.103993 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:30:57.103993 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:30:57.103993 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:30:57.103993 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:57.103993 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:57.103993 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:57.103993 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:57.103993 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:57.103993 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:57.103993 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:57.103993 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:57.103993 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:57.103993 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:57.103993 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:57.103993 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:57.104941 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:57.104941 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:57.104941 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:30:57.104941 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:30:57.104941 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:30:57.104941 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:30:57.104941 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:30:57.104941 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:30:57.104941 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 14:30:57.104941 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:57.104941 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 548/1091] 10.222.0.30 () {56 vars in 1093 bytes} [Tue Aug 30 14:30:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:30:57.109764 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df35f0f-74f3-4f3a-a4c8-93f2844a7726 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:57.205329 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faee74b-2bad-41f8-88e6-7096ce14971c None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:57.205660 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faee74b-2bad-41f8-88e6-7096ce14971c None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:57.205886 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faee74b-2bad-41f8-88e6-7096ce14971c None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:57.207386 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 549/1092] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:30:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:30:57.208908 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df35f0f-74f3-4f3a-a4c8-93f2844a7726 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:57.212200 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae2de9-2dec-493f-86d2-a25f99c200dc None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:57.212480 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae2de9-2dec-493f-86d2-a25f99c200dc None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:57.212751 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae2de9-2dec-493f-86d2-a25f99c200dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:57.226347 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:57.231252 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-18ae2de9-2dec-493f-86d2-a25f99c200dc None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:30:57.232456 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-18ae2de9-2dec-493f-86d2-a25f99c200dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:30:57.245350 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7026943-f187-40c2-863c-fc0b8e0f0b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:57.245962 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7026943-f187-40c2-863c-fc0b8e0f0b8e None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:57.246241 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7026943-f187-40c2-863c-fc0b8e0f0b8e None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:57.246528 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7026943-f187-40c2-863c-fc0b8e0f0b8e None admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:57.247286 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7026943-f187-40c2-863c-fc0b8e0f0b8e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:57.247596 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7026943-f187-40c2-863c-fc0b8e0f0b8e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:57.249023 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7026943-f187-40c2-863c-fc0b8e0f0b8e None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:57.267226 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 544/1093] 10.222.0.30 () {56 vars in 1133 bytes} [Tue Aug 30 14:30:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:30:57.273063 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7026943-f187-40c2-863c-fc0b8e0f0b8e None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:57.300315 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 550/1094] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:30:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:30:57.372451 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7026943-f187-40c2-863c-fc0b8e0f0b8e None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:57.405668 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb076db8-1203-48a5-ba70-7868aeafe722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:57.406265 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb076db8-1203-48a5-ba70-7868aeafe722 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:57.406544 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb076db8-1203-48a5-ba70-7868aeafe722 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:57.406814 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb076db8-1203-48a5-ba70-7868aeafe722 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:57.449256 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:57.474481 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb076db8-1203-48a5-ba70-7868aeafe722 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:57.475955 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb076db8-1203-48a5-ba70-7868aeafe722 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:57.563081 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ae2de9-2dec-493f-86d2-a25f99c200dc None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:57.572948 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 545/1095] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:30:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 14:30:57.578557 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb076db8-1203-48a5-ba70-7868aeafe722 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:57.650812 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:57.661511 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ae2de9-2dec-493f-86d2-a25f99c200dc None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:57.677545 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb076db8-1203-48a5-ba70-7868aeafe722 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:57.679857 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:57.698108 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9cc06c-b7c1-430b-9388-fe9f9eca6ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:57.698737 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9cc06c-b7c1-430b-9388-fe9f9eca6ac3 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:57.699014 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9cc06c-b7c1-430b-9388-fe9f9eca6ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:57.699316 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9cc06c-b7c1-430b-9388-fe9f9eca6ac3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:57.709158 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9cc06c-b7c1-430b-9388-fe9f9eca6ac3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:30:57.711190 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc31538-8a36-47fd-aa74-b6b181c44708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:57.711938 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc31538-8a36-47fd-aa74-b6b181c44708 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:57.712215 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc31538-8a36-47fd-aa74-b6b181c44708 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:57.712482 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc31538-8a36-47fd-aa74-b6b181c44708 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:57.714211 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-9f9cc06c-b7c1-430b-9388-fe9f9eca6ac3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:30:57.714211 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:57.714211 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:57.714211 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:57.714211 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:57.714211 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:57.714211 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:57.714211 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:57.714211 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:57.714211 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:57.714211 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:57.714211 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:57.714211 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:30:57.714211 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:30:57.714211 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:30:57.714211 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:30:57.715351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:30:57.715351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:30:57.715351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:30:57.715351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:30:57.715351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:57.715351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:57.715351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:57.715351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:57.715351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:57.715351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:57.715351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:57.715351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:57.715351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:57.715351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:57.715351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:57.715351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:57.716382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:57.716382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:57.716382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:57.716382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:57.716382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:30:57.716382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:30:57.716382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:30:57.716382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:30:57.716382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:30:57.716382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:30:57.716382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:30:57.716382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:30:57.716382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 551/1096] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 30 14:30:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:30:57.721153 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9cc06c-b7c1-430b-9388-fe9f9eca6ac3 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:57.726545 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc31538-8a36-47fd-aa74-b6b181c44708 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:57.728557 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc31538-8a36-47fd-aa74-b6b181c44708 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:57.743825 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-bfc31538-8a36-47fd-aa74-b6b181c44708 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:57.743825 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:57.743825 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:57.743825 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:57.743825 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:57.743825 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:57.743825 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:57.743825 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:57.743825 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:57.743825 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:57.743825 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:57.743825 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:57.743825 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:30:57.743825 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:30:57.743825 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:30:57.743825 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:30:57.745087 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:57.745087 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:57.745087 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:57.745087 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:57.745087 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:57.745087 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:57.745087 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:57.745087 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:57.745087 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:57.745087 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:57.745087 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:57.745087 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:57.745087 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:57.745087 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:57.745087 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:57.745087 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:57.745087 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:30:57.746178 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:30:57.746178 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:30:57.746178 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:57.746178 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 546/1097] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:30:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:30:57.750852 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc31538-8a36-47fd-aa74-b6b181c44708 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:57.822905 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9cc06c-b7c1-430b-9388-fe9f9eca6ac3 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:57.847618 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc31538-8a36-47fd-aa74-b6b181c44708 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:57.857639 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc7cb7f-5cb0-406f-b8fd-2ac35315dbb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:57.858238 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc7cb7f-5cb0-406f-b8fd-2ac35315dbb1 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:57.858518 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc7cb7f-5cb0-406f-b8fd-2ac35315dbb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:57.858786 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc7cb7f-5cb0-406f-b8fd-2ac35315dbb1 None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:57.859468 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc7cb7f-5cb0-406f-b8fd-2ac35315dbb1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:57.859764 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc7cb7f-5cb0-406f-b8fd-2ac35315dbb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:57.861121 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc7cb7f-5cb0-406f-b8fd-2ac35315dbb1 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:57.869308 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 552/1098] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:30:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:30:57.873361 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:57.874987 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc7cb7f-5cb0-406f-b8fd-2ac35315dbb1 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:57.882721 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3970ea4-42e5-48d8-adae-70263e45d7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:57.883358 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3970ea4-42e5-48d8-adae-70263e45d7dd None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:57.883666 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3970ea4-42e5-48d8-adae-70263e45d7dd None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:57.883930 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3970ea4-42e5-48d8-adae-70263e45d7dd None admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:57.884746 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3970ea4-42e5-48d8-adae-70263e45d7dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:57.885043 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3970ea4-42e5-48d8-adae-70263e45d7dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:57.886397 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3970ea4-42e5-48d8-adae-70263e45d7dd None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:57.898160 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 547/1099] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:30:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:30:57.904081 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3970ea4-42e5-48d8-adae-70263e45d7dd None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:57.918458 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:57.980561 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc7cb7f-5cb0-406f-b8fd-2ac35315dbb1 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:58.004995 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3970ea4-42e5-48d8-adae-70263e45d7dd None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:58.014459 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dc90b1-a2ab-4a97-8dd3-51f9ce890706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:58.015086 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc90b1-a2ab-4a97-8dd3-51f9ce890706 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:58.015404 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc90b1-a2ab-4a97-8dd3-51f9ce890706 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:58.015672 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc90b1-a2ab-4a97-8dd3-51f9ce890706 None admin] PATH_INFO: `/v3/users/73cae25f4b724e7e908b1138f58a66f8` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:58.016808 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dc90b1-a2ab-4a97-8dd3-51f9ce890706 None admin] RBAC: Authorizing `identity:get_user(user_id=73cae25f4b724e7e908b1138f58a66f8)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:58.018230 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dc90b1-a2ab-4a97-8dd3-51f9ce890706 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:58.019845 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 553/1100] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:30:57 2022] GET /identity/v3/users/73cae25f4b724e7e908b1138f58a66f8 => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:30:58.026211 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dc90b1-a2ab-4a97-8dd3-51f9ce890706 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:58.039197 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24e1dca-6d4d-43bb-a336-7362f900e01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:58.040551 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e1dca-6d4d-43bb-a336-7362f900e01c None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:58.040832 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e1dca-6d4d-43bb-a336-7362f900e01c None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:58.041100 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e1dca-6d4d-43bb-a336-7362f900e01c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:58.047732 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24e1dca-6d4d-43bb-a336-7362f900e01c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:58.049193 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24e1dca-6d4d-43bb-a336-7362f900e01c None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:58.060690 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-b24e1dca-6d4d-43bb-a336-7362f900e01c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:30:58.060690 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:30:58.060690 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:30:58.060690 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:30:58.060690 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:30:58.060690 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:30:58.060690 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:30:58.060690 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:30:58.060690 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:30:58.060690 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:30:58.060690 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:30:58.060690 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:30:58.060690 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:30:58.060690 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:30:58.060690 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:30:58.061593 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:30:58.061593 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:30:58.061593 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:30:58.061593 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:30:58.061593 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:30:58.061593 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:30:58.061593 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:30:58.061593 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:30:58.061593 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:30:58.061593 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:30:58.061593 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:30:58.061593 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:30:58.061593 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:30:58.061593 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:30:58.061593 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:30:58.061593 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:30:58.061593 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:30:58.062672 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:30:58.062672 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:30:58.062672 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:30:58.062672 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:30:58.062672 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:30:58.062672 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:30:58.062672 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:30:58.062672 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:30:58.062672 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 548/1101] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:30:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:30:58.067465 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24e1dca-6d4d-43bb-a336-7362f900e01c None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:58.113219 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:58.135582 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dc90b1-a2ab-4a97-8dd3-51f9ce890706 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:58.144336 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:58.168626 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24e1dca-6d4d-43bb-a336-7362f900e01c None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:58.170021 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97978f45-f878-4da5-9ac6-5f6f36942a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:58.170619 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97978f45-f878-4da5-9ac6-5f6f36942a6a None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:58.170904 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97978f45-f878-4da5-9ac6-5f6f36942a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:58.171179 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97978f45-f878-4da5-9ac6-5f6f36942a6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:58.205291 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca216de-2a26-4fa5-810b-60e1a9080162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:58.205899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca216de-2a26-4fa5-810b-60e1a9080162 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:58.206198 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca216de-2a26-4fa5-810b-60e1a9080162 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:58.206452 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca216de-2a26-4fa5-810b-60e1a9080162 None admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:58.207147 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca216de-2a26-4fa5-810b-60e1a9080162 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:58.207480 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca216de-2a26-4fa5-810b-60e1a9080162 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:58.208916 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca216de-2a26-4fa5-810b-60e1a9080162 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:58.222007 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 549/1102] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 30 14:30:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:30:58.227647 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca216de-2a26-4fa5-810b-60e1a9080162 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:58.244846 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97978f45-f878-4da5-9ac6-5f6f36942a6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:58.246297 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97978f45-f878-4da5-9ac6-5f6f36942a6a None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:58.326049 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca216de-2a26-4fa5-810b-60e1a9080162 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:58.333135 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:58.348591 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 554/1103] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:30:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:30:58.354525 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97978f45-f878-4da5-9ac6-5f6f36942a6a None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:58.360596 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd5f162-6db3-4fca-aa55-1969f6fa1d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:58.361196 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd5f162-6db3-4fca-aa55-1969f6fa1d51 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:58.361467 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd5f162-6db3-4fca-aa55-1969f6fa1d51 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:58.361734 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd5f162-6db3-4fca-aa55-1969f6fa1d51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:58.362451 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd5f162-6db3-4fca-aa55-1969f6fa1d51 None admin] RBAC: Adding query filter params (role.id=2345dd1fbf154d6888e8113c51914966, scope.project.id=01cf71ca5abf4576b89d10c85af6d2de, user.id=8878de66a03044e38e507028853a0cd1) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:58.362749 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd5f162-6db3-4fca-aa55-1969f6fa1d51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:58.364007 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:30:58.364192 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd5f162-6db3-4fca-aa55-1969f6fa1d51 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:58.370962 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 550/1104] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:30:58 2022] GET /identity/v3/role_assignments?user.id=8878de66a03044e38e507028853a0cd1&scope.project.id=01cf71ca5abf4576b89d10c85af6d2de&role.id=2345dd1fbf154d6888e8113c51914966 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:30:58.460824 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97978f45-f878-4da5-9ac6-5f6f36942a6a None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:58.496916 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd284782-13d8-469f-b283-ebe2bb563ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:58.497562 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd284782-13d8-469f-b283-ebe2bb563ccb None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:58.497844 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd284782-13d8-469f-b283-ebe2bb563ccb None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:58.498119 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd284782-13d8-469f-b283-ebe2bb563ccb None admin] PATH_INFO: `/v3/projects/ba3cb001c0304cb3bbefeded2355d23f` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:58.499335 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd284782-13d8-469f-b283-ebe2bb563ccb None admin] RBAC: Authorizing `identity:get_project(project_id=ba3cb001c0304cb3bbefeded2355d23f)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:58.500792 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd284782-13d8-469f-b283-ebe2bb563ccb None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:58.502487 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 555/1105] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:30:58 2022] GET /identity/v3/projects/ba3cb001c0304cb3bbefeded2355d23f => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:30:58.509160 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd5f162-6db3-4fca-aa55-1969f6fa1d51 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:30:58.613478 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd5f162-6db3-4fca-aa55-1969f6fa1d51 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:30:58.648119 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d0055d-173f-4dbb-9f03-1f0dabe7fba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:30:58.648654 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0055d-173f-4dbb-9f03-1f0dabe7fba5 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:30:58.648934 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0055d-173f-4dbb-9f03-1f0dabe7fba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:30:58.649359 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0055d-173f-4dbb-9f03-1f0dabe7fba5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:30:58.650540 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d0055d-173f-4dbb-9f03-1f0dabe7fba5 None admin] RBAC: Adding query filter params (role.id=3cda0615d87d4af0860a2a5e9b4e2c05, scope.project.id=ba3cb001c0304cb3bbefeded2355d23f, user.id=73cae25f4b724e7e908b1138f58a66f8) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:30:58.650885 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d0055d-173f-4dbb-9f03-1f0dabe7fba5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:30:58.652361 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d0055d-173f-4dbb-9f03-1f0dabe7fba5 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:30:58.663296 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 551/1106] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:30:58 2022] GET /identity/v3/role_assignments?user.id=73cae25f4b724e7e908b1138f58a66f8&scope.project.id=ba3cb001c0304cb3bbefeded2355d23f&role.id=3cda0615d87d4af0860a2a5e9b4e2c05 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:00.598466 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f3d87-d972-493f-a518-ded53e03e07b None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:00.598981 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f3d87-d972-493f-a518-ded53e03e07b None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:00.599073 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f3d87-d972-493f-a518-ded53e03e07b None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:00.600392 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 556/1107] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:31:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:00.606371 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba7964f-c1c7-4c6b-85dc-63493bad8f40 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:00.606762 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba7964f-c1c7-4c6b-85dc-63493bad8f40 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:00.606920 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba7964f-c1c7-4c6b-85dc-63493bad8f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:00.625487 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-7ba7964f-c1c7-4c6b-85dc-63493bad8f40 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:00.626664 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-7ba7964f-c1c7-4c6b-85dc-63493bad8f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:00.652856 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:00.697902 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 552/1108] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:31:00.963251 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450f3d87-d972-493f-a518-ded53e03e07b None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:01.038861 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:01.063569 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450f3d87-d972-493f-a518-ded53e03e07b None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:01.096956 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2328af3e-2923-467c-8432-71bd69d19654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:01.097583 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328af3e-2923-467c-8432-71bd69d19654 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:01.097860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328af3e-2923-467c-8432-71bd69d19654 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:01.098130 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328af3e-2923-467c-8432-71bd69d19654 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:01.099255 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2328af3e-2923-467c-8432-71bd69d19654 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:01.100688 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2328af3e-2923-467c-8432-71bd69d19654 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:01.102809 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 557/1109] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:01.108561 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba7964f-c1c7-4c6b-85dc-63493bad8f40 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:01.205802 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba7964f-c1c7-4c6b-85dc-63493bad8f40 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:01.239391 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d4e943-fbc2-42c1-96e1-3888d15bbc0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:01.239936 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4e943-fbc2-42c1-96e1-3888d15bbc0a None admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:01.240213 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4e943-fbc2-42c1-96e1-3888d15bbc0a None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:01.240494 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4e943-fbc2-42c1-96e1-3888d15bbc0a None admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:01.241128 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d4e943-fbc2-42c1-96e1-3888d15bbc0a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:01.242545 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d4e943-fbc2-42c1-96e1-3888d15bbc0a None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:01.263919 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 553/1110] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 30 14:31:01 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:31:03.218675 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a9eedd-9aa5-4ed5-87b4-32d2ab3d85d9 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:03.219168 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a9eedd-9aa5-4ed5-87b4-32d2ab3d85d9 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:03.219301 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a9eedd-9aa5-4ed5-87b4-32d2ab3d85d9 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:03.220641 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 558/1111] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:31:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:03.225807 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab880d35-7775-47e3-b7cf-01818c9d53c1 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:03.226199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab880d35-7775-47e3-b7cf-01818c9d53c1 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:03.226348 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab880d35-7775-47e3-b7cf-01818c9d53c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:03.244688 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-ab880d35-7775-47e3-b7cf-01818c9d53c1 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:03.245851 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-ab880d35-7775-47e3-b7cf-01818c9d53c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:03.260132 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:03.312770 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 554/1112] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:31:03.581354 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a9eedd-9aa5-4ed5-87b4-32d2ab3d85d9 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:03.682814 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a9eedd-9aa5-4ed5-87b4-32d2ab3d85d9 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:03.716031 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc0e83-c4a0-4047-ba67-ff79751ccac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:03.716642 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc0e83-c4a0-4047-ba67-ff79751ccac1 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:03.716918 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc0e83-c4a0-4047-ba67-ff79751ccac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:03.717187 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc0e83-c4a0-4047-ba67-ff79751ccac1 None admin] PATH_INFO: `/v3/roles/1c4886a7a7fd402d8221a0158ddcb167` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:03.718554 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc0e83-c4a0-4047-ba67-ff79751ccac1 None admin] RBAC: Authorizing `identity:get_role(role_id=1c4886a7a7fd402d8221a0158ddcb167)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:03.719936 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc0e83-c4a0-4047-ba67-ff79751ccac1 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:03.721245 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 559/1113] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:31:03 2022] GET /identity/v3/roles/1c4886a7a7fd402d8221a0158ddcb167 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:03.727584 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab880d35-7775-47e3-b7cf-01818c9d53c1 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:03.824046 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab880d35-7775-47e3-b7cf-01818c9d53c1 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:03.856795 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f448f-810c-4faa-b549-5243825aead9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:03.857355 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f448f-810c-4faa-b549-5243825aead9 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:03.857631 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f448f-810c-4faa-b549-5243825aead9 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:03.857903 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f448f-810c-4faa-b549-5243825aead9 None admin] PATH_INFO: `/v3/users/2940aca6007d4fe8bb9ca51d85d1944e` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:03.871146 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1f448f-810c-4faa-b549-5243825aead9 None admin] RBAC: Authorizing `identity:get_user(user_id=2940aca6007d4fe8bb9ca51d85d1944e)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:03.872612 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1f448f-810c-4faa-b549-5243825aead9 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:03.874187 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 555/1114] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:31:03 2022] GET /identity/v3/users/2940aca6007d4fe8bb9ca51d85d1944e => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:03.879948 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc0e83-c4a0-4047-ba67-ff79751ccac1 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:03.894999 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:03.977851 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc0e83-c4a0-4047-ba67-ff79751ccac1 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:04.011173 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df7e25f-9d26-4128-a6fd-8d1a07cff321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:04.011794 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df7e25f-9d26-4128-a6fd-8d1a07cff321 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:04.012075 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df7e25f-9d26-4128-a6fd-8d1a07cff321 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:04.012349 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df7e25f-9d26-4128-a6fd-8d1a07cff321 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:04.077707 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df7e25f-9d26-4128-a6fd-8d1a07cff321 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:04.079138 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df7e25f-9d26-4128-a6fd-8d1a07cff321 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:04.092035 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:04.174643 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 560/1115] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:31:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:31:04.180932 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f448f-810c-4faa-b549-5243825aead9 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:04.191624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:04.278494 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f448f-810c-4faa-b549-5243825aead9 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:04.311731 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd88cd-cee7-4034-887a-10616efc3776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:04.312346 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd88cd-cee7-4034-887a-10616efc3776 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:04.312620 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd88cd-cee7-4034-887a-10616efc3776 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:04.312889 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd88cd-cee7-4034-887a-10616efc3776 None admin] PATH_INFO: `/v3/projects/ba3cb001c0304cb3bbefeded2355d23f` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:04.314016 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecd88cd-cee7-4034-887a-10616efc3776 None admin] RBAC: Authorizing `identity:get_project(project_id=ba3cb001c0304cb3bbefeded2355d23f)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:04.315540 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecd88cd-cee7-4034-887a-10616efc3776 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:04.317124 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 556/1116] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:31:04 2022] GET /identity/v3/projects/ba3cb001c0304cb3bbefeded2355d23f => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:04.323913 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df7e25f-9d26-4128-a6fd-8d1a07cff321 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:04.419639 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df7e25f-9d26-4128-a6fd-8d1a07cff321 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:04.432967 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:04.453725 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3edcfc1-4dd8-4b98-a49a-3239fc40d13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:04.454327 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3edcfc1-4dd8-4b98-a49a-3239fc40d13a None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:04.454606 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3edcfc1-4dd8-4b98-a49a-3239fc40d13a None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:04.454878 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3edcfc1-4dd8-4b98-a49a-3239fc40d13a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:04.455639 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3edcfc1-4dd8-4b98-a49a-3239fc40d13a None admin] RBAC: Adding query filter params (role.id=1c4886a7a7fd402d8221a0158ddcb167, scope.project.id=ba3cb001c0304cb3bbefeded2355d23f, user.id=2940aca6007d4fe8bb9ca51d85d1944e) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:04.455950 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3edcfc1-4dd8-4b98-a49a-3239fc40d13a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:04.457366 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3edcfc1-4dd8-4b98-a49a-3239fc40d13a None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:04.462634 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 561/1117] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:31:04 2022] GET /identity/v3/role_assignments?user.id=2940aca6007d4fe8bb9ca51d85d1944e&scope.project.id=ba3cb001c0304cb3bbefeded2355d23f&role.id=1c4886a7a7fd402d8221a0158ddcb167 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:06.324838 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fbd803-3472-40bc-b46c-8b04c02f8bd7 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:06.325382 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fbd803-3472-40bc-b46c-8b04c02f8bd7 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:06.325382 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fbd803-3472-40bc-b46c-8b04c02f8bd7 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:06.326724 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 557/1118] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:31:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:06.331833 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395635e-73bd-4cd1-99cc-62d2897a76dc None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:06.332211 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395635e-73bd-4cd1-99cc-62d2897a76dc None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:06.332382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395635e-73bd-4cd1-99cc-62d2897a76dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:06.350820 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-3395635e-73bd-4cd1-99cc-62d2897a76dc None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:06.352067 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-3395635e-73bd-4cd1-99cc-62d2897a76dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:06.416112 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 562/1119] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:31:06.684917 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fbd803-3472-40bc-b46c-8b04c02f8bd7 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:06.760364 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:06.784427 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fbd803-3472-40bc-b46c-8b04c02f8bd7 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:06.818218 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2935d61d-6982-4f67-b5fe-81330776d376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:06.818837 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2935d61d-6982-4f67-b5fe-81330776d376 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:06.819116 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2935d61d-6982-4f67-b5fe-81330776d376 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:06.819417 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2935d61d-6982-4f67-b5fe-81330776d376 None admin] PATH_INFO: `/v3/roles/1c4886a7a7fd402d8221a0158ddcb167` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:06.820903 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2935d61d-6982-4f67-b5fe-81330776d376 None admin] RBAC: Authorizing `identity:get_role(role_id=1c4886a7a7fd402d8221a0158ddcb167)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:06.822268 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2935d61d-6982-4f67-b5fe-81330776d376 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:06.823621 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 558/1120] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:31:06 2022] GET /identity/v3/roles/1c4886a7a7fd402d8221a0158ddcb167 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:06.829410 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3395635e-73bd-4cd1-99cc-62d2897a76dc None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:06.926342 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3395635e-73bd-4cd1-99cc-62d2897a76dc None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:06.933474 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:06.960107 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88530af0-afd9-490a-bfd7-d8b9c9508236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:06.960667 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88530af0-afd9-490a-bfd7-d8b9c9508236 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:06.960946 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88530af0-afd9-490a-bfd7-d8b9c9508236 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:06.961218 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88530af0-afd9-490a-bfd7-d8b9c9508236 None admin] PATH_INFO: `/v3/users/2940aca6007d4fe8bb9ca51d85d1944e` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:06.962872 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88530af0-afd9-490a-bfd7-d8b9c9508236 None admin] RBAC: Authorizing `identity:get_user(user_id=2940aca6007d4fe8bb9ca51d85d1944e)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:06.964326 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88530af0-afd9-490a-bfd7-d8b9c9508236 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:06.965893 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 563/1121] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:31:06 2022] GET /identity/v3/users/2940aca6007d4fe8bb9ca51d85d1944e => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:06.971735 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2935d61d-6982-4f67-b5fe-81330776d376 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:07.071037 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2935d61d-6982-4f67-b5fe-81330776d376 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:07.105139 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9d88c9-188f-44c2-bd0d-4883edeed9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:07.105751 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d88c9-188f-44c2-bd0d-4883edeed9b5 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:07.106027 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d88c9-188f-44c2-bd0d-4883edeed9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:07.106295 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d88c9-188f-44c2-bd0d-4883edeed9b5 None admin] PATH_INFO: `/v3/projects/ba3cb001c0304cb3bbefeded2355d23f` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:07.107428 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9d88c9-188f-44c2-bd0d-4883edeed9b5 None admin] RBAC: Authorizing `identity:get_project(project_id=ba3cb001c0304cb3bbefeded2355d23f)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:07.108838 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9d88c9-188f-44c2-bd0d-4883edeed9b5 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:07.110441 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 559/1122] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:31:06 2022] GET /identity/v3/projects/ba3cb001c0304cb3bbefeded2355d23f => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:07.116297 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88530af0-afd9-490a-bfd7-d8b9c9508236 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:07.213153 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88530af0-afd9-490a-bfd7-d8b9c9508236 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:07.246738 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b34b1a6-8c80-4bf2-b246-44adbde60f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:07.247410 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34b1a6-8c80-4bf2-b246-44adbde60f59 None admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:07.247687 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34b1a6-8c80-4bf2-b246-44adbde60f59 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:07.247955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34b1a6-8c80-4bf2-b246-44adbde60f59 None admin] PATH_INFO: `/v3/projects/ba3cb001c0304cb3bbefeded2355d23f/users/2940aca6007d4fe8bb9ca51d85d1944e/roles/1c4886a7a7fd402d8221a0158ddcb167` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:07.250047 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b34b1a6-8c80-4bf2-b246-44adbde60f59 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba3cb001c0304cb3bbefeded2355d23f, user_id=2940aca6007d4fe8bb9ca51d85d1944e, role_id=1c4886a7a7fd402d8221a0158ddcb167)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:07.251482 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b34b1a6-8c80-4bf2-b246-44adbde60f59 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:07.264239 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 564/1123] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 30 14:31:07 2022] PUT /identity/v3/projects/ba3cb001c0304cb3bbefeded2355d23f/users/2940aca6007d4fe8bb9ca51d85d1944e/roles/1c4886a7a7fd402d8221a0158ddcb167 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:31:09.140819 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a15339a-2188-4efd-8a5b-862be4443225 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:09.141386 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a15339a-2188-4efd-8a5b-862be4443225 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:09.141386 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a15339a-2188-4efd-8a5b-862be4443225 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:09.142702 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 560/1124] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:31:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:09.147744 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e5f78-a85c-4783-bcf2-88fc1a474fb3 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:09.148119 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e5f78-a85c-4783-bcf2-88fc1a474fb3 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:09.148291 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e5f78-a85c-4783-bcf2-88fc1a474fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:09.167150 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-204e5f78-a85c-4783-bcf2-88fc1a474fb3 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:09.168372 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-204e5f78-a85c-4783-bcf2-88fc1a474fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:09.194720 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:09.235740 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 565/1125] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:31:09.505666 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a15339a-2188-4efd-8a5b-862be4443225 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:09.520686 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:09.606725 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a15339a-2188-4efd-8a5b-862be4443225 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:09.639872 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c662257-246a-4fe3-819f-3ee0692a5fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:09.640491 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c662257-246a-4fe3-819f-3ee0692a5fb0 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:09.640772 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c662257-246a-4fe3-819f-3ee0692a5fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:09.641050 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c662257-246a-4fe3-819f-3ee0692a5fb0 None admin] PATH_INFO: `/v3/roles/1c4886a7a7fd402d8221a0158ddcb167` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:09.642441 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c662257-246a-4fe3-819f-3ee0692a5fb0 None admin] RBAC: Authorizing `identity:get_role(role_id=1c4886a7a7fd402d8221a0158ddcb167)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:09.643837 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c662257-246a-4fe3-819f-3ee0692a5fb0 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:09.645154 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 561/1126] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:31:09 2022] GET /identity/v3/roles/1c4886a7a7fd402d8221a0158ddcb167 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:09.651999 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204e5f78-a85c-4783-bcf2-88fc1a474fb3 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:09.747942 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204e5f78-a85c-4783-bcf2-88fc1a474fb3 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:09.781112 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192e7c95-4945-48a2-b5a3-a773cae99bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:09.781669 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e7c95-4945-48a2-b5a3-a773cae99bbb None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:09.781946 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e7c95-4945-48a2-b5a3-a773cae99bbb None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:09.782216 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e7c95-4945-48a2-b5a3-a773cae99bbb None admin] PATH_INFO: `/v3/users/2940aca6007d4fe8bb9ca51d85d1944e` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:09.783351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192e7c95-4945-48a2-b5a3-a773cae99bbb None admin] RBAC: Authorizing `identity:get_user(user_id=2940aca6007d4fe8bb9ca51d85d1944e)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:09.784767 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192e7c95-4945-48a2-b5a3-a773cae99bbb None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:09.786328 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 566/1127] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:31:09 2022] GET /identity/v3/users/2940aca6007d4fe8bb9ca51d85d1944e => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:09.792437 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c662257-246a-4fe3-819f-3ee0692a5fb0 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:09.871008 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:09.890389 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c662257-246a-4fe3-819f-3ee0692a5fb0 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:09.922931 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e914c4e-6873-4b9d-a468-2205d91825fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:09.923549 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e914c4e-6873-4b9d-a468-2205d91825fc None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:09.923826 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e914c4e-6873-4b9d-a468-2205d91825fc None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:09.924097 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e914c4e-6873-4b9d-a468-2205d91825fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:09.989314 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e914c4e-6873-4b9d-a468-2205d91825fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:09.990745 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e914c4e-6873-4b9d-a468-2205d91825fc None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:10.067129 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:10.086956 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 562/1128] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:31:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:31:10.092898 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192e7c95-4945-48a2-b5a3-a773cae99bbb None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:10.123906 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:10.191457 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192e7c95-4945-48a2-b5a3-a773cae99bbb None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:10.225180 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b269d9-0470-4332-819a-3fe3ce664648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:10.225803 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b269d9-0470-4332-819a-3fe3ce664648 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:10.226079 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b269d9-0470-4332-819a-3fe3ce664648 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:10.226346 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b269d9-0470-4332-819a-3fe3ce664648 None admin] PATH_INFO: `/v3/projects/ba3cb001c0304cb3bbefeded2355d23f` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:10.228007 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b269d9-0470-4332-819a-3fe3ce664648 None admin] RBAC: Authorizing `identity:get_project(project_id=ba3cb001c0304cb3bbefeded2355d23f)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:10.229429 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b269d9-0470-4332-819a-3fe3ce664648 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:10.231106 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 567/1129] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:31:10 2022] GET /identity/v3/projects/ba3cb001c0304cb3bbefeded2355d23f => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:10.237212 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e914c4e-6873-4b9d-a468-2205d91825fc None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:10.335045 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e914c4e-6873-4b9d-a468-2205d91825fc None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:10.368048 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a3bb0-cabf-4ede-92f5-3edc623e773f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:10.368635 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a3bb0-cabf-4ede-92f5-3edc623e773f None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:10.368917 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a3bb0-cabf-4ede-92f5-3edc623e773f None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:10.369191 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a3bb0-cabf-4ede-92f5-3edc623e773f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:10.369908 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440a3bb0-cabf-4ede-92f5-3edc623e773f None admin] RBAC: Adding query filter params (role.id=1c4886a7a7fd402d8221a0158ddcb167, scope.project.id=ba3cb001c0304cb3bbefeded2355d23f, user.id=2940aca6007d4fe8bb9ca51d85d1944e) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:10.370217 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440a3bb0-cabf-4ede-92f5-3edc623e773f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:10.371669 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440a3bb0-cabf-4ede-92f5-3edc623e773f None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:10.377224 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 563/1130] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:31:10 2022] GET /identity/v3/role_assignments?user.id=2940aca6007d4fe8bb9ca51d85d1944e&scope.project.id=ba3cb001c0304cb3bbefeded2355d23f&role.id=1c4886a7a7fd402d8221a0158ddcb167 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:12.279395 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c79fef-f64e-4640-899f-43792a914e75 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:12.279883 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c79fef-f64e-4640-899f-43792a914e75 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:12.279993 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c79fef-f64e-4640-899f-43792a914e75 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:12.281271 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 568/1131] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:31:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:12.286668 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0add91-46cd-4ab0-a3ea-6aa8901743db None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:12.287077 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0add91-46cd-4ab0-a3ea-6aa8901743db None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:12.287227 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0add91-46cd-4ab0-a3ea-6aa8901743db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:12.307740 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-bb0add91-46cd-4ab0-a3ea-6aa8901743db None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:12.308912 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-bb0add91-46cd-4ab0-a3ea-6aa8901743db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:12.348629 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:12.374758 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 564/1132] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:31:12.642663 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c79fef-f64e-4640-899f-43792a914e75 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:12.739047 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:12.743950 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c79fef-f64e-4640-899f-43792a914e75 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:12.776640 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32614c4-4c7b-4406-840d-3cd9a6840bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:12.777255 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32614c4-4c7b-4406-840d-3cd9a6840bf1 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:12.777533 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32614c4-4c7b-4406-840d-3cd9a6840bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:12.777804 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32614c4-4c7b-4406-840d-3cd9a6840bf1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:12.778904 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32614c4-4c7b-4406-840d-3cd9a6840bf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:12.780352 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32614c4-4c7b-4406-840d-3cd9a6840bf1 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:12.781901 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 569/1133] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:12.787645 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0add91-46cd-4ab0-a3ea-6aa8901743db None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:12.884873 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0add91-46cd-4ab0-a3ea-6aa8901743db None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:12.918176 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad827f6-d281-4ea6-aee6-0b76849fdd20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:12.918707 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad827f6-d281-4ea6-aee6-0b76849fdd20 None admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:12.918985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad827f6-d281-4ea6-aee6-0b76849fdd20 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:12.919285 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad827f6-d281-4ea6-aee6-0b76849fdd20 None admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:12.919942 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad827f6-d281-4ea6-aee6-0b76849fdd20 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:12.921354 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad827f6-d281-4ea6-aee6-0b76849fdd20 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:12.948372 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 565/1134] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Aug 30 14:31:12 2022] POST /identity/v3/projects => generated 311 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:31:14.854623 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5614a4a8-0077-46ca-9308-a68915db67ce None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:14.855182 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5614a4a8-0077-46ca-9308-a68915db67ce None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:14.855182 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5614a4a8-0077-46ca-9308-a68915db67ce None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:14.856546 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 570/1135] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:31:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:14.861610 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65240680-c22b-4957-bded-84e262dcf0ec None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:14.862028 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65240680-c22b-4957-bded-84e262dcf0ec None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:14.862163 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65240680-c22b-4957-bded-84e262dcf0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:14.880755 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-65240680-c22b-4957-bded-84e262dcf0ec None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:14.881953 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-65240680-c22b-4957-bded-84e262dcf0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:14.903047 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:14.949017 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 566/1136] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:31:15.220070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5614a4a8-0077-46ca-9308-a68915db67ce None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:15.318482 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5614a4a8-0077-46ca-9308-a68915db67ce None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:15.351708 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6f45a7-db05-4b48-9784-0834b6daf72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:15.352322 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f45a7-db05-4b48-9784-0834b6daf72a None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:15.352593 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f45a7-db05-4b48-9784-0834b6daf72a None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:15.352857 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f45a7-db05-4b48-9784-0834b6daf72a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:15.353945 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6f45a7-db05-4b48-9784-0834b6daf72a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:15.355383 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6f45a7-db05-4b48-9784-0834b6daf72a None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:15.356929 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 571/1137] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:31:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:15.362697 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65240680-c22b-4957-bded-84e262dcf0ec None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:15.458183 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65240680-c22b-4957-bded-84e262dcf0ec None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:15.491180 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8cc817-b055-4b48-a89c-5e1259aa9046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:15.491746 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8cc817-b055-4b48-a89c-5e1259aa9046 None admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:15.492020 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8cc817-b055-4b48-a89c-5e1259aa9046 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:15.492913 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8cc817-b055-4b48-a89c-5e1259aa9046 None admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:15.493560 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8cc817-b055-4b48-a89c-5e1259aa9046 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:15.494965 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8cc817-b055-4b48-a89c-5e1259aa9046 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:15.514904 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 567/1138] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 30 14:31:15 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:31:17.437597 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e00909-55e9-49b5-9f1b-e53c1908f457 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:17.438139 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e00909-55e9-49b5-9f1b-e53c1908f457 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:17.438139 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e00909-55e9-49b5-9f1b-e53c1908f457 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:17.439507 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 572/1139] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:31:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:17.444668 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f833b-6782-4dd7-a847-5c4dd98049ef None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:17.445033 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f833b-6782-4dd7-a847-5c4dd98049ef None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:17.445223 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f833b-6782-4dd7-a847-5c4dd98049ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:17.463710 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-3c7f833b-6782-4dd7-a847-5c4dd98049ef None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:17.464831 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-3c7f833b-6782-4dd7-a847-5c4dd98049ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:17.471955 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:17.531117 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 568/1140] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:31:17.797537 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e00909-55e9-49b5-9f1b-e53c1908f457 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:17.832812 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:17.896967 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e00909-55e9-49b5-9f1b-e53c1908f457 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:17.930237 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9ad893-4d98-4607-90bb-595d9c88c100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:17.930852 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ad893-4d98-4607-90bb-595d9c88c100 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:17.931127 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ad893-4d98-4607-90bb-595d9c88c100 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:17.931419 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ad893-4d98-4607-90bb-595d9c88c100 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:17.941332 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9ad893-4d98-4607-90bb-595d9c88c100 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:31:17.946160 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-db9ad893-4d98-4607-90bb-595d9c88c100 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:31:17.946160 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:17.946160 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:17.946160 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:17.946160 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:17.946160 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:17.946160 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:17.946160 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:17.946160 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:17.946160 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:17.946160 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:17.946160 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:17.946160 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:31:17.946160 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:31:17.946160 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:31:17.946160 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:31:17.947339 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:31:17.947339 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:31:17.947339 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:31:17.947339 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:31:17.947339 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:17.947339 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:17.947339 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:17.947339 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:17.947339 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:17.947339 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:17.947339 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:17.947339 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:17.947339 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:17.947339 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:17.947339 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:17.947339 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:17.948552 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:17.948552 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:17.948552 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:17.948552 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:17.948552 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:31:17.948552 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:31:17.948552 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:31:17.948552 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:31:17.948552 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:31:17.948552 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:31:17.948552 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:31:17.948552 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:31:17.948552 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 573/1141] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 30 14:31:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:31:17.953306 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f833b-6782-4dd7-a847-5c4dd98049ef None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:18.048429 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f833b-6782-4dd7-a847-5c4dd98049ef None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:18.081543 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce79961f-def6-453f-b454-92d08c482f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:18.082137 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce79961f-def6-453f-b454-92d08c482f32 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:18.082414 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce79961f-def6-453f-b454-92d08c482f32 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:18.082685 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce79961f-def6-453f-b454-92d08c482f32 None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:18.083367 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce79961f-def6-453f-b454-92d08c482f32 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:18.083668 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce79961f-def6-453f-b454-92d08c482f32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:18.085028 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce79961f-def6-453f-b454-92d08c482f32 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:18.093621 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 569/1142] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:31:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:18.099291 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9ad893-4d98-4607-90bb-595d9c88c100 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:18.192144 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:18.198059 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9ad893-4d98-4607-90bb-595d9c88c100 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:18.232437 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bdb79a-ee68-41eb-90b6-d4c4a387e485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:18.233057 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bdb79a-ee68-41eb-90b6-d4c4a387e485 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:18.233336 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bdb79a-ee68-41eb-90b6-d4c4a387e485 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:18.233607 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bdb79a-ee68-41eb-90b6-d4c4a387e485 None admin] PATH_INFO: `/v3/users/c218d9debd154c3fb2eafa732f2e3bf7` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:18.246538 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bdb79a-ee68-41eb-90b6-d4c4a387e485 None admin] RBAC: Authorizing `identity:get_user(user_id=c218d9debd154c3fb2eafa732f2e3bf7)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:18.247993 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bdb79a-ee68-41eb-90b6-d4c4a387e485 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:18.249594 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 574/1143] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:31:18 2022] GET /identity/v3/users/c218d9debd154c3fb2eafa732f2e3bf7 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:18.255483 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce79961f-def6-453f-b454-92d08c482f32 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:18.259698 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:18.354737 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce79961f-def6-453f-b454-92d08c482f32 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:18.386666 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6919f23c-9d5d-4f2b-95a3-f499550f25ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:18.387291 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919f23c-9d5d-4f2b-95a3-f499550f25ad None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:18.387570 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919f23c-9d5d-4f2b-95a3-f499550f25ad None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:18.387839 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919f23c-9d5d-4f2b-95a3-f499550f25ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:18.453323 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6919f23c-9d5d-4f2b-95a3-f499550f25ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:18.454750 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6919f23c-9d5d-4f2b-95a3-f499550f25ad None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:18.457230 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:18.550111 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 570/1144] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:31:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:31:18.556126 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bdb79a-ee68-41eb-90b6-d4c4a387e485 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:18.652744 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bdb79a-ee68-41eb-90b6-d4c4a387e485 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:18.685593 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64ed49c-402d-47b3-bf4d-3ecc76a6b308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:18.686213 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64ed49c-402d-47b3-bf4d-3ecc76a6b308 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:18.686493 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64ed49c-402d-47b3-bf4d-3ecc76a6b308 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:18.686768 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64ed49c-402d-47b3-bf4d-3ecc76a6b308 None admin] PATH_INFO: `/v3/projects/235d7c0a459e435fabfb302e6a6698ba` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:18.687895 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64ed49c-402d-47b3-bf4d-3ecc76a6b308 None admin] RBAC: Authorizing `identity:get_project(project_id=235d7c0a459e435fabfb302e6a6698ba)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:18.689318 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64ed49c-402d-47b3-bf4d-3ecc76a6b308 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:18.690933 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 575/1145] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:31:18 2022] GET /identity/v3/projects/235d7c0a459e435fabfb302e6a6698ba => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:18.697419 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6919f23c-9d5d-4f2b-95a3-f499550f25ad None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:18.792270 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6919f23c-9d5d-4f2b-95a3-f499550f25ad None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:18.794501 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:18.825551 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7201ba8-1201-4108-b7b8-486e627d0064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:18.826138 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7201ba8-1201-4108-b7b8-486e627d0064 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:18.826416 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7201ba8-1201-4108-b7b8-486e627d0064 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:18.826685 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7201ba8-1201-4108-b7b8-486e627d0064 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:18.827434 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7201ba8-1201-4108-b7b8-486e627d0064 None admin] RBAC: Adding query filter params (role.id=3cda0615d87d4af0860a2a5e9b4e2c05, scope.project.id=235d7c0a459e435fabfb302e6a6698ba, user.id=c218d9debd154c3fb2eafa732f2e3bf7) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:18.827738 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7201ba8-1201-4108-b7b8-486e627d0064 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:18.829141 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7201ba8-1201-4108-b7b8-486e627d0064 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:18.834340 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 571/1146] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:31:18 2022] GET /identity/v3/role_assignments?user.id=c218d9debd154c3fb2eafa732f2e3bf7&scope.project.id=235d7c0a459e435fabfb302e6a6698ba&role.id=3cda0615d87d4af0860a2a5e9b4e2c05 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:20.696491 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9a49a-042a-434b-a0e9-38269bc6d2be None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:20.697027 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9a49a-042a-434b-a0e9-38269bc6d2be None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:20.697027 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9a49a-042a-434b-a0e9-38269bc6d2be None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:20.698640 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 576/1147] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:31:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:20.704161 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5349b94-c2f9-4584-9626-9ca5304f3280 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:20.704597 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5349b94-c2f9-4584-9626-9ca5304f3280 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:20.704690 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5349b94-c2f9-4584-9626-9ca5304f3280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:20.722325 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-e5349b94-c2f9-4584-9626-9ca5304f3280 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:20.723558 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-e5349b94-c2f9-4584-9626-9ca5304f3280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:20.788122 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 572/1148] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:31:21.056507 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9a49a-042a-434b-a0e9-38269bc6d2be None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:21.080689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:21.158634 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9a49a-042a-434b-a0e9-38269bc6d2be None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:21.191922 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb7d89-1867-473c-b5b9-1405a4b1975a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:21.192537 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb7d89-1867-473c-b5b9-1405a4b1975a None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:21.192819 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb7d89-1867-473c-b5b9-1405a4b1975a None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:21.193089 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb7d89-1867-473c-b5b9-1405a4b1975a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:21.203566 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eb7d89-1867-473c-b5b9-1405a4b1975a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:31:21.208330 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-04eb7d89-1867-473c-b5b9-1405a4b1975a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:31:21.208330 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:21.208330 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:21.208330 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:21.208330 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:21.208330 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:21.208330 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:21.208330 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:21.208330 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:21.208330 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:21.208330 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:21.208330 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:21.208330 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:31:21.208330 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:31:21.208330 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:31:21.208330 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:31:21.209338 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:31:21.209338 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:31:21.209338 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:31:21.209338 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:31:21.209338 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:21.209338 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:21.209338 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:21.209338 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:21.209338 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:21.209338 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:21.209338 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:21.209338 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:21.209338 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:21.209338 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:21.209338 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:21.209338 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:21.210215 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:21.210215 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:21.210215 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:21.210215 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:21.210215 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:31:21.210215 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:31:21.210215 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:31:21.210215 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:31:21.210215 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:31:21.210215 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:31:21.210215 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:31:21.210215 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:31:21.210215 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 577/1149] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 30 14:31:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:21.215699 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5349b94-c2f9-4584-9626-9ca5304f3280 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:21.308262 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:21.312336 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5349b94-c2f9-4584-9626-9ca5304f3280 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:21.345797 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f761e5-4910-4720-a8a0-a5f3bf415935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:21.346397 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f761e5-4910-4720-a8a0-a5f3bf415935 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:21.346676 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f761e5-4910-4720-a8a0-a5f3bf415935 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:21.346946 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f761e5-4910-4720-a8a0-a5f3bf415935 None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:21.347634 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f761e5-4910-4720-a8a0-a5f3bf415935 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:21.347937 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f761e5-4910-4720-a8a0-a5f3bf415935 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:21.349290 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f761e5-4910-4720-a8a0-a5f3bf415935 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:21.357242 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 573/1150] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:31:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:21.363052 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb7d89-1867-473c-b5b9-1405a4b1975a None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:21.444065 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:21.462608 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb7d89-1867-473c-b5b9-1405a4b1975a None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:21.496234 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd957e9-ca77-42cd-a18a-71cf5812cfd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:21.496853 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd957e9-ca77-42cd-a18a-71cf5812cfd0 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:21.497160 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd957e9-ca77-42cd-a18a-71cf5812cfd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:21.497432 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd957e9-ca77-42cd-a18a-71cf5812cfd0 None admin] PATH_INFO: `/v3/users/c218d9debd154c3fb2eafa732f2e3bf7` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:21.498546 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd957e9-ca77-42cd-a18a-71cf5812cfd0 None admin] RBAC: Authorizing `identity:get_user(user_id=c218d9debd154c3fb2eafa732f2e3bf7)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:21.500064 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd957e9-ca77-42cd-a18a-71cf5812cfd0 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:21.501633 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 578/1151] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:31:21 2022] GET /identity/v3/users/c218d9debd154c3fb2eafa732f2e3bf7 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:21.507526 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f761e5-4910-4720-a8a0-a5f3bf415935 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:21.603994 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f761e5-4910-4720-a8a0-a5f3bf415935 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:21.636320 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace6001b-dc6b-4ffb-b8d8-ae969ef891bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:21.636930 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6001b-dc6b-4ffb-b8d8-ae969ef891bb None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:21.637206 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6001b-dc6b-4ffb-b8d8-ae969ef891bb None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:21.637475 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6001b-dc6b-4ffb-b8d8-ae969ef891bb None admin] PATH_INFO: `/v3/projects/235d7c0a459e435fabfb302e6a6698ba` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:21.638576 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace6001b-dc6b-4ffb-b8d8-ae969ef891bb None admin] RBAC: Authorizing `identity:get_project(project_id=235d7c0a459e435fabfb302e6a6698ba)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:21.640015 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace6001b-dc6b-4ffb-b8d8-ae969ef891bb None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:21.641596 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 574/1152] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:31:21 2022] GET /identity/v3/projects/235d7c0a459e435fabfb302e6a6698ba => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:21.647461 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd957e9-ca77-42cd-a18a-71cf5812cfd0 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:21.745638 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd957e9-ca77-42cd-a18a-71cf5812cfd0 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:21.779322 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2a97a2-eca4-4ff7-8668-6761606ed8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:21.779961 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a97a2-eca4-4ff7-8668-6761606ed8d6 None admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:21.780237 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a97a2-eca4-4ff7-8668-6761606ed8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:21.780510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a97a2-eca4-4ff7-8668-6761606ed8d6 None admin] PATH_INFO: `/v3/projects/235d7c0a459e435fabfb302e6a6698ba/users/c218d9debd154c3fb2eafa732f2e3bf7/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:21.782388 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2a97a2-eca4-4ff7-8668-6761606ed8d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=235d7c0a459e435fabfb302e6a6698ba, user_id=c218d9debd154c3fb2eafa732f2e3bf7, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:21.783830 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2a97a2-eca4-4ff7-8668-6761606ed8d6 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:21.797036 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 579/1153] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 30 14:31:21 2022] PUT /identity/v3/projects/235d7c0a459e435fabfb302e6a6698ba/users/c218d9debd154c3fb2eafa732f2e3bf7/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:31:23.685510 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c70ef-d0b9-4834-8753-1acf7d3b4f89 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:23.686056 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c70ef-d0b9-4834-8753-1acf7d3b4f89 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:23.686056 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c70ef-d0b9-4834-8753-1acf7d3b4f89 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:23.687407 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 575/1154] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:31:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:23.692435 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd6136a-fd3e-480d-8508-91d7fef9af3a None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:23.692817 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd6136a-fd3e-480d-8508-91d7fef9af3a None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:23.692979 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd6136a-fd3e-480d-8508-91d7fef9af3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:23.711545 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-dbd6136a-fd3e-480d-8508-91d7fef9af3a None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:23.712729 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-dbd6136a-fd3e-480d-8508-91d7fef9af3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:23.715459 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:23.778044 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 580/1155] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:31:24.044167 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6c70ef-d0b9-4834-8753-1acf7d3b4f89 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:24.072601 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:24.144473 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6c70ef-d0b9-4834-8753-1acf7d3b4f89 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:24.176919 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a651738-d7c0-4a64-98d9-4e113335736f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:24.177530 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a651738-d7c0-4a64-98d9-4e113335736f None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:24.177801 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a651738-d7c0-4a64-98d9-4e113335736f None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:24.178064 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a651738-d7c0-4a64-98d9-4e113335736f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:24.188792 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a651738-d7c0-4a64-98d9-4e113335736f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:31:24.193408 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-3a651738-d7c0-4a64-98d9-4e113335736f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:31:24.193408 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:24.193408 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:24.193408 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:24.193408 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:24.193408 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:24.193408 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:24.193408 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:24.193408 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:24.193408 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:24.193408 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:24.193408 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:24.193408 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:31:24.193408 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:31:24.193408 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:31:24.193408 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:31:24.194478 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:31:24.194478 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:31:24.194478 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:31:24.194478 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:31:24.194478 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:24.194478 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:24.194478 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:24.194478 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:24.194478 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:24.194478 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:24.194478 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:24.194478 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:24.194478 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:24.194478 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:24.194478 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:24.194478 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:24.195701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:24.195701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:24.195701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:24.195701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:24.195701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:31:24.195701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:31:24.195701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:31:24.195701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:31:24.195701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:31:24.195701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:31:24.195701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:31:24.195701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:31:24.195701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 576/1156] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 30 14:31:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:24.200784 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd6136a-fd3e-480d-8508-91d7fef9af3a None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:24.296820 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd6136a-fd3e-480d-8508-91d7fef9af3a None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:24.330564 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92543d3c-43d7-43a6-948c-d0368329c7c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:24.331157 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92543d3c-43d7-43a6-948c-d0368329c7c9 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:24.331459 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92543d3c-43d7-43a6-948c-d0368329c7c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:24.331727 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92543d3c-43d7-43a6-948c-d0368329c7c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:24.332377 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92543d3c-43d7-43a6-948c-d0368329c7c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:24.332676 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92543d3c-43d7-43a6-948c-d0368329c7c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:24.334026 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92543d3c-43d7-43a6-948c-d0368329c7c9 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:24.342705 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 581/1157] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:31:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:24.348583 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a651738-d7c0-4a64-98d9-4e113335736f None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:24.436118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:24.448507 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a651738-d7c0-4a64-98d9-4e113335736f None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:24.480728 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba4da5b-507b-48fb-a9d8-3ac6ffa29e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:24.481346 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4da5b-507b-48fb-a9d8-3ac6ffa29e51 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:24.481623 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4da5b-507b-48fb-a9d8-3ac6ffa29e51 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:24.481894 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4da5b-507b-48fb-a9d8-3ac6ffa29e51 None admin] PATH_INFO: `/v3/users/c218d9debd154c3fb2eafa732f2e3bf7` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:24.483004 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba4da5b-507b-48fb-a9d8-3ac6ffa29e51 None admin] RBAC: Authorizing `identity:get_user(user_id=c218d9debd154c3fb2eafa732f2e3bf7)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:24.484452 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba4da5b-507b-48fb-a9d8-3ac6ffa29e51 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:24.486042 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 577/1158] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:31:24 2022] GET /identity/v3/users/c218d9debd154c3fb2eafa732f2e3bf7 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:24.491979 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92543d3c-43d7-43a6-948c-d0368329c7c9 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:24.494821 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:24.591371 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92543d3c-43d7-43a6-948c-d0368329c7c9 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:24.623187 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5114a5-84c5-4b70-bedb-752decd8ddc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:24.623804 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5114a5-84c5-4b70-bedb-752decd8ddc4 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:24.624083 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5114a5-84c5-4b70-bedb-752decd8ddc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:24.624355 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5114a5-84c5-4b70-bedb-752decd8ddc4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:24.689476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5114a5-84c5-4b70-bedb-752decd8ddc4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:24.690908 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5114a5-84c5-4b70-bedb-752decd8ddc4 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:24.692650 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:24.786986 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 582/1159] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:31:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:31:24.793365 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba4da5b-507b-48fb-a9d8-3ac6ffa29e51 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:24.892808 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba4da5b-507b-48fb-a9d8-3ac6ffa29e51 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:24.924888 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13f60b9-2d5b-4419-b7e0-1b39512dda04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:24.925511 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13f60b9-2d5b-4419-b7e0-1b39512dda04 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:24.925791 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13f60b9-2d5b-4419-b7e0-1b39512dda04 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:24.926067 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13f60b9-2d5b-4419-b7e0-1b39512dda04 None admin] PATH_INFO: `/v3/projects/235d7c0a459e435fabfb302e6a6698ba` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:24.927560 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13f60b9-2d5b-4419-b7e0-1b39512dda04 None admin] RBAC: Authorizing `identity:get_project(project_id=235d7c0a459e435fabfb302e6a6698ba)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:24.928988 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13f60b9-2d5b-4419-b7e0-1b39512dda04 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:24.931269 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 578/1160] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:31:24 2022] GET /identity/v3/projects/235d7c0a459e435fabfb302e6a6698ba => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:24.937641 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5114a5-84c5-4b70-bedb-752decd8ddc4 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:25.034594 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5114a5-84c5-4b70-bedb-752decd8ddc4 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:25.036122 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:25.069275 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c53eb75-5c20-45ba-ab9c-7e53bf611cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:25.069868 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53eb75-5c20-45ba-ab9c-7e53bf611cac None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:25.070189 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53eb75-5c20-45ba-ab9c-7e53bf611cac None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:25.070462 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53eb75-5c20-45ba-ab9c-7e53bf611cac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:25.071182 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c53eb75-5c20-45ba-ab9c-7e53bf611cac None admin] RBAC: Adding query filter params (role.id=3cda0615d87d4af0860a2a5e9b4e2c05, scope.project.id=235d7c0a459e435fabfb302e6a6698ba, user.id=c218d9debd154c3fb2eafa732f2e3bf7) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:25.071525 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c53eb75-5c20-45ba-ab9c-7e53bf611cac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:25.072941 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c53eb75-5c20-45ba-ab9c-7e53bf611cac None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:25.078542 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 583/1161] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:31:24 2022] GET /identity/v3/role_assignments?user.id=c218d9debd154c3fb2eafa732f2e3bf7&scope.project.id=235d7c0a459e435fabfb302e6a6698ba&role.id=3cda0615d87d4af0860a2a5e9b4e2c05 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:26.995126 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f85432-05b2-4b09-bc91-558619943076 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:26.995701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f85432-05b2-4b09-bc91-558619943076 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:26.995701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f85432-05b2-4b09-bc91-558619943076 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:26.997066 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 579/1162] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:31:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:27.002370 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e9332-9994-41e8-89b2-290c89a916ce None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:27.002788 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e9332-9994-41e8-89b2-290c89a916ce None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:27.002904 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e9332-9994-41e8-89b2-290c89a916ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:27.020853 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-365e9332-9994-41e8-89b2-290c89a916ce None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:27.022022 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-365e9332-9994-41e8-89b2-290c89a916ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:27.087083 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 584/1163] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:31:27.357490 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f85432-05b2-4b09-bc91-558619943076 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:27.382612 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:27.458950 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f85432-05b2-4b09-bc91-558619943076 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:27.491626 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0bcfee-5224-4ad2-a5db-977a981b9122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:27.492787 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0bcfee-5224-4ad2-a5db-977a981b9122 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:27.493058 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0bcfee-5224-4ad2-a5db-977a981b9122 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:27.493323 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0bcfee-5224-4ad2-a5db-977a981b9122 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:27.559978 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0bcfee-5224-4ad2-a5db-977a981b9122 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:27.561399 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0bcfee-5224-4ad2-a5db-977a981b9122 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:27.584787 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:27.664364 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 580/1164] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:31:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:31:27.669807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365e9332-9994-41e8-89b2-290c89a916ce None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:27.760524 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:27.766264 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365e9332-9994-41e8-89b2-290c89a916ce None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:27.799037 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679919dd-08a5-4c6b-9710-83446803ef6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:27.799679 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679919dd-08a5-4c6b-9710-83446803ef6c None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:27.799957 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679919dd-08a5-4c6b-9710-83446803ef6c None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:27.800223 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679919dd-08a5-4c6b-9710-83446803ef6c None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:27.806577 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679919dd-08a5-4c6b-9710-83446803ef6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:27.808026 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679919dd-08a5-4c6b-9710-83446803ef6c None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:27.819228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-679919dd-08a5-4c6b-9710-83446803ef6c None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 30 14:31:27.819228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:27.819228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 30 14:31:27.819228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 30 14:31:27.819228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:31:27.819228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:31:27.819228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:31:27.819228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:31:27.819228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 30 14:31:27.819228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:31:27.819228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:31:27.819228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:31:27.819228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:27.819228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:27.819228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:27.819228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:27.819228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:27.821471 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:27.821471 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:27.821471 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:27.821471 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:27.821471 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:27.821471 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:27.821471 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:31:27.821471 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:31:27.821471 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:31:27.821471 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:31:27.821471 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:27.821471 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:27.821471 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:27.821471 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:27.821471 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:27.821471 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:27.822616 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:27.822616 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:27.822616 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:27.822616 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:27.822616 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:27.822616 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:27.822616 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:27.822616 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:27.822616 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:27.822616 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:27.822616 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 30 14:31:27.822616 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:31:27.822616 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 30 14:31:27.822616 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:31:27.822616 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 585/1165] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Aug 30 14:31:27 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:27.826810 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0bcfee-5224-4ad2-a5db-977a981b9122 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:27.923946 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0bcfee-5224-4ad2-a5db-977a981b9122 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:27.948821 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:27.959562 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0062199-35a4-4e97-a956-30dceef1940b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:27.960150 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0062199-35a4-4e97-a956-30dceef1940b None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:27.960423 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0062199-35a4-4e97-a956-30dceef1940b None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:27.961021 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0062199-35a4-4e97-a956-30dceef1940b None admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:27.961435 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0062199-35a4-4e97-a956-30dceef1940b None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:27.961731 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0062199-35a4-4e97-a956-30dceef1940b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:27.963065 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0062199-35a4-4e97-a956-30dceef1940b None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:27.969890 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 581/1166] 10.222.0.30 () {56 vars in 1115 bytes} [Tue Aug 30 14:31:27 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:29.840710 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99272d-9de8-4ab7-8380-569adf431005 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:29.841242 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99272d-9de8-4ab7-8380-569adf431005 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:29.841242 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99272d-9de8-4ab7-8380-569adf431005 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:29.842581 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 586/1167] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:31:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:29.847683 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb5c40-d911-400c-9fee-40c393fd88ca None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:29.848096 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb5c40-d911-400c-9fee-40c393fd88ca None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:29.848237 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb5c40-d911-400c-9fee-40c393fd88ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:29.866462 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-b6bb5c40-d911-400c-9fee-40c393fd88ca None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:29.868417 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-b6bb5c40-d911-400c-9fee-40c393fd88ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:29.933246 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 582/1168] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:31:30.196910 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd99272d-9de8-4ab7-8380-569adf431005 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:30.294980 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd99272d-9de8-4ab7-8380-569adf431005 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:30.329214 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4f1071-7b37-4aee-9e56-2c1d889487ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:30.329796 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f1071-7b37-4aee-9e56-2c1d889487ff None admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:30.330067 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f1071-7b37-4aee-9e56-2c1d889487ff None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:30.330330 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f1071-7b37-4aee-9e56-2c1d889487ff None admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:30.330978 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4f1071-7b37-4aee-9e56-2c1d889487ff None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:30.332419 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4f1071-7b37-4aee-9e56-2c1d889487ff None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:30.347780 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 587/1169] 10.222.0.30 () {60 vars in 1138 bytes} [Tue Aug 30 14:31:30 2022] POST /identity/v3/domains => generated 267 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:31:32.245597 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c5a716-a9b7-4bec-b0da-9b364c8ab896 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:32.246065 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c5a716-a9b7-4bec-b0da-9b364c8ab896 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:32.246178 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c5a716-a9b7-4bec-b0da-9b364c8ab896 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:32.247494 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 583/1170] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:31:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:32.252638 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2720e15-4a5d-46ca-a837-08991f063368 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:32.253047 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2720e15-4a5d-46ca-a837-08991f063368 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:32.253186 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2720e15-4a5d-46ca-a837-08991f063368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:32.271499 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-b2720e15-4a5d-46ca-a837-08991f063368 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:32.272678 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-b2720e15-4a5d-46ca-a837-08991f063368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:32.283717 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:32.339620 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 588/1171] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:31:32.618521 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c5a716-a9b7-4bec-b0da-9b364c8ab896 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:32.716683 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c5a716-a9b7-4bec-b0da-9b364c8ab896 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:32.733771 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:32.750768 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ce5aea-67f4-4726-8210-8ed4302aa2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:32.751412 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce5aea-67f4-4726-8210-8ed4302aa2e1 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:32.751693 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce5aea-67f4-4726-8210-8ed4302aa2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:32.751965 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce5aea-67f4-4726-8210-8ed4302aa2e1 None admin] PATH_INFO: `/v3/domains/f8fc22c48b4048deb427bb7a4ffa5c33` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:32.762279 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ce5aea-67f4-4726-8210-8ed4302aa2e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f8fc22c48b4048deb427bb7a4ffa5c33)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:32.763756 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ce5aea-67f4-4726-8210-8ed4302aa2e1 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:32.765330 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 584/1172] 10.222.0.30 () {56 vars in 1183 bytes} [Tue Aug 30 14:31:32 2022] GET /identity/v3/domains/f8fc22c48b4048deb427bb7a4ffa5c33 => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:32.771091 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2720e15-4a5d-46ca-a837-08991f063368 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:32.868162 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2720e15-4a5d-46ca-a837-08991f063368 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:32.900356 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4011046d-c2b1-4fad-9340-87e6d809da81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:32.900898 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011046d-c2b1-4fad-9340-87e6d809da81 None admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:32.901178 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011046d-c2b1-4fad-9340-87e6d809da81 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:32.901448 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011046d-c2b1-4fad-9340-87e6d809da81 None admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:32.902080 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4011046d-c2b1-4fad-9340-87e6d809da81 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:32.903531 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4011046d-c2b1-4fad-9340-87e6d809da81 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:32.923032 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:32.930725 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 589/1173] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Aug 30 14:31:32 2022] POST /identity/v3/projects => generated 361 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:31:34.871980 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b4da36-e1ab-419f-80b7-166d6d3fea44 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:34.872575 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b4da36-e1ab-419f-80b7-166d6d3fea44 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:34.872575 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b4da36-e1ab-419f-80b7-166d6d3fea44 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:34.873850 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 585/1174] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:31:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:34.879062 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af5847-09ab-4159-a664-7ca1d8c8f8e5 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:34.879484 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af5847-09ab-4159-a664-7ca1d8c8f8e5 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:34.879643 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af5847-09ab-4159-a664-7ca1d8c8f8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:34.898285 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-b9af5847-09ab-4159-a664-7ca1d8c8f8e5 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:34.899914 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-b9af5847-09ab-4159-a664-7ca1d8c8f8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:34.973497 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 590/1175] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:31:35.245203 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b4da36-e1ab-419f-80b7-166d6d3fea44 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:35.344621 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b4da36-e1ab-419f-80b7-166d6d3fea44 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:35.376777 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ce061a-ec84-45ca-9294-fb15d57507c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:35.377388 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce061a-ec84-45ca-9294-fb15d57507c6 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:35.377665 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce061a-ec84-45ca-9294-fb15d57507c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:35.377934 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce061a-ec84-45ca-9294-fb15d57507c6 None admin] PATH_INFO: `/v3/domains/f8fc22c48b4048deb427bb7a4ffa5c33` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:35.379042 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ce061a-ec84-45ca-9294-fb15d57507c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f8fc22c48b4048deb427bb7a4ffa5c33)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:35.380482 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ce061a-ec84-45ca-9294-fb15d57507c6 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:35.382029 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 586/1176] 10.222.0.30 () {56 vars in 1183 bytes} [Tue Aug 30 14:31:35 2022] GET /identity/v3/domains/f8fc22c48b4048deb427bb7a4ffa5c33 => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:35.388324 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af5847-09ab-4159-a664-7ca1d8c8f8e5 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:35.484284 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af5847-09ab-4159-a664-7ca1d8c8f8e5 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:35.510416 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:35.518263 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83566920-e35c-46f3-bf8d-b521d13d2713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:35.518857 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83566920-e35c-46f3-bf8d-b521d13d2713 None admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:35.519133 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83566920-e35c-46f3-bf8d-b521d13d2713 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:35.519430 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83566920-e35c-46f3-bf8d-b521d13d2713 None admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:35.520072 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83566920-e35c-46f3-bf8d-b521d13d2713 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:35.521489 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83566920-e35c-46f3-bf8d-b521d13d2713 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:35.541041 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 591/1177] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 30 14:31:35 2022] POST /identity/v3/users => generated 311 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:31:37.466402 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dceb66-73d0-4ce7-988f-222a62dfb496 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:37.466913 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dceb66-73d0-4ce7-988f-222a62dfb496 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:37.466913 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dceb66-73d0-4ce7-988f-222a62dfb496 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:37.468238 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 587/1178] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:31:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:37.473299 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7675a-8ed2-4b5d-8ac2-ea177d23f6c7 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:37.473689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7675a-8ed2-4b5d-8ac2-ea177d23f6c7 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:37.473848 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7675a-8ed2-4b5d-8ac2-ea177d23f6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:37.492615 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-5fc7675a-8ed2-4b5d-8ac2-ea177d23f6c7 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:37.493789 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-5fc7675a-8ed2-4b5d-8ac2-ea177d23f6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:37.558767 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 592/1179] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:31:37.832693 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dceb66-73d0-4ce7-988f-222a62dfb496 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:37.880401 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:37.934708 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dceb66-73d0-4ce7-988f-222a62dfb496 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:37.968461 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96acfade-4fb9-465f-950c-bf9420006d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:37.969073 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96acfade-4fb9-465f-950c-bf9420006d92 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:37.969348 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96acfade-4fb9-465f-950c-bf9420006d92 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:37.969613 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96acfade-4fb9-465f-950c-bf9420006d92 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:37.980300 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96acfade-4fb9-465f-950c-bf9420006d92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:31:37.984966 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-96acfade-4fb9-465f-950c-bf9420006d92 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:31:37.984966 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:37.984966 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:37.984966 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:37.984966 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:37.984966 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:37.984966 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:37.984966 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:37.984966 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:37.984966 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:37.984966 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:37.984966 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:37.984966 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:31:37.984966 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:31:37.984966 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:31:37.984966 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:31:37.986118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:31:37.986118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:31:37.986118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:31:37.986118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:31:37.986118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:37.986118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:37.986118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:37.986118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:37.986118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:37.986118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:37.986118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:37.986118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:37.986118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:37.986118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:37.986118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:37.986118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:37.987285 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:37.987285 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:37.987285 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:37.987285 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:37.987285 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:31:37.987285 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:31:37.987285 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:31:37.987285 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:31:37.987285 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:31:37.987285 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:31:37.987285 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:31:37.987285 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:31:37.987285 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 588/1180] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 30 14:31:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:37.992238 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc7675a-8ed2-4b5d-8ac2-ea177d23f6c7 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:38.089825 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc7675a-8ed2-4b5d-8ac2-ea177d23f6c7 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:38.104438 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:38.125010 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18692f03-03ff-40ef-92b5-633f24c1b579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:38.125605 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18692f03-03ff-40ef-92b5-633f24c1b579 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:38.125884 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18692f03-03ff-40ef-92b5-633f24c1b579 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:38.126155 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18692f03-03ff-40ef-92b5-633f24c1b579 None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:38.126813 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18692f03-03ff-40ef-92b5-633f24c1b579 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:38.127124 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18692f03-03ff-40ef-92b5-633f24c1b579 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:38.128518 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18692f03-03ff-40ef-92b5-633f24c1b579 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:38.136455 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 593/1181] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:31:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:38.142221 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96acfade-4fb9-465f-950c-bf9420006d92 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:38.240427 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96acfade-4fb9-465f-950c-bf9420006d92 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:38.249135 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:38.275318 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4ca8da-2a21-4475-a2ad-784d99fd21aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:38.275941 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ca8da-2a21-4475-a2ad-784d99fd21aa None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:38.276222 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ca8da-2a21-4475-a2ad-784d99fd21aa None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:38.276494 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ca8da-2a21-4475-a2ad-784d99fd21aa None admin] PATH_INFO: `/v3/users/f4fb1d1852974392855fd2d72ad6e6ee` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:38.290720 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4ca8da-2a21-4475-a2ad-784d99fd21aa None admin] RBAC: Authorizing `identity:get_user(user_id=f4fb1d1852974392855fd2d72ad6e6ee)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:38.292187 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4ca8da-2a21-4475-a2ad-784d99fd21aa None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:38.293781 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 589/1182] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:31:38 2022] GET /identity/v3/users/f4fb1d1852974392855fd2d72ad6e6ee => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:38.300367 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18692f03-03ff-40ef-92b5-633f24c1b579 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:38.398994 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18692f03-03ff-40ef-92b5-633f24c1b579 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:38.431734 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07823632-a7ca-423d-b6c2-1eb54ff126c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:38.432325 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07823632-a7ca-423d-b6c2-1eb54ff126c8 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:38.432606 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07823632-a7ca-423d-b6c2-1eb54ff126c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:38.432878 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07823632-a7ca-423d-b6c2-1eb54ff126c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:38.476476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:38.500189 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07823632-a7ca-423d-b6c2-1eb54ff126c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:38.501620 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07823632-a7ca-423d-b6c2-1eb54ff126c8 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:38.596904 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 594/1183] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:31:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:31:38.603742 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4ca8da-2a21-4475-a2ad-784d99fd21aa None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:38.700357 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4ca8da-2a21-4475-a2ad-784d99fd21aa None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:38.733009 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa02a2fb-8066-4f87-b2f7-de83f3aa1bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:38.733624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa02a2fb-8066-4f87-b2f7-de83f3aa1bde None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:38.733901 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa02a2fb-8066-4f87-b2f7-de83f3aa1bde None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:38.734170 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa02a2fb-8066-4f87-b2f7-de83f3aa1bde None admin] PATH_INFO: `/v3/projects/7258eea809404d33870bfbc834958c03` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:38.735300 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa02a2fb-8066-4f87-b2f7-de83f3aa1bde None admin] RBAC: Authorizing `identity:get_project(project_id=7258eea809404d33870bfbc834958c03)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:38.736705 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa02a2fb-8066-4f87-b2f7-de83f3aa1bde None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:38.738312 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 590/1184] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:31:38 2022] GET /identity/v3/projects/7258eea809404d33870bfbc834958c03 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:38.744385 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07823632-a7ca-423d-b6c2-1eb54ff126c8 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:38.817837 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:38.841580 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07823632-a7ca-423d-b6c2-1eb54ff126c8 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:38.873834 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441db2b2-9996-448d-9e83-78d33bcae783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:38.874424 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441db2b2-9996-448d-9e83-78d33bcae783 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:38.874701 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441db2b2-9996-448d-9e83-78d33bcae783 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:38.874971 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441db2b2-9996-448d-9e83-78d33bcae783 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:38.875716 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441db2b2-9996-448d-9e83-78d33bcae783 None admin] RBAC: Adding query filter params (role.id=3cda0615d87d4af0860a2a5e9b4e2c05, scope.project.id=7258eea809404d33870bfbc834958c03, user.id=f4fb1d1852974392855fd2d72ad6e6ee) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:38.876022 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441db2b2-9996-448d-9e83-78d33bcae783 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:38.877430 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441db2b2-9996-448d-9e83-78d33bcae783 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:38.883459 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 595/1185] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:31:38 2022] GET /identity/v3/role_assignments?user.id=f4fb1d1852974392855fd2d72ad6e6ee&scope.project.id=7258eea809404d33870bfbc834958c03&role.id=3cda0615d87d4af0860a2a5e9b4e2c05 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:40.739221 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b823588-4ecb-4f89-88d6-7f1edc44b514 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:40.739773 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b823588-4ecb-4f89-88d6-7f1edc44b514 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:40.739773 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b823588-4ecb-4f89-88d6-7f1edc44b514 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:40.741102 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 591/1186] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:31:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:40.746848 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c911c-cde9-420e-8c46-719c10a9b146 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:40.747261 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c911c-cde9-420e-8c46-719c10a9b146 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:40.747410 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c911c-cde9-420e-8c46-719c10a9b146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:40.765617 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-9a7c911c-cde9-420e-8c46-719c10a9b146 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:40.766784 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-9a7c911c-cde9-420e-8c46-719c10a9b146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:40.830555 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 596/1187] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:31:41.099300 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b823588-4ecb-4f89-88d6-7f1edc44b514 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:41.131049 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:41.200249 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b823588-4ecb-4f89-88d6-7f1edc44b514 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:41.233999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb848d-484e-4548-945f-a8d613fde154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:41.234629 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb848d-484e-4548-945f-a8d613fde154 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:41.234908 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb848d-484e-4548-945f-a8d613fde154 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:41.235177 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb848d-484e-4548-945f-a8d613fde154 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:41.245762 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbb848d-484e-4548-945f-a8d613fde154 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:31:41.250502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-dcbb848d-484e-4548-945f-a8d613fde154 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:31:41.250502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:41.250502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:41.250502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:41.250502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:41.250502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:41.250502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:41.250502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:41.250502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:41.250502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:41.250502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:41.250502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:41.250502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:31:41.250502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:31:41.250502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:31:41.250502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:31:41.251608 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:31:41.251608 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:31:41.251608 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:31:41.251608 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:31:41.251608 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:41.251608 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:41.251608 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:41.251608 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:41.251608 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:41.251608 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:41.251608 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:41.251608 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:41.251608 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:41.251608 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:41.251608 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:41.251608 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:41.252646 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:41.252646 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:41.252646 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:41.252646 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:41.252646 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:31:41.252646 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:31:41.252646 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:31:41.252646 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:31:41.252646 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:31:41.252646 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:31:41.252646 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:31:41.252646 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:31:41.252646 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 592/1188] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 30 14:31:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:41.257700 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7c911c-cde9-420e-8c46-719c10a9b146 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:41.325877 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:41.357277 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7c911c-cde9-420e-8c46-719c10a9b146 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:41.389539 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca95a5b-f9d5-440d-8924-510c9fa8bf0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:41.390132 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca95a5b-f9d5-440d-8924-510c9fa8bf0d None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:41.390409 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca95a5b-f9d5-440d-8924-510c9fa8bf0d None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:41.390679 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca95a5b-f9d5-440d-8924-510c9fa8bf0d None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:41.391362 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca95a5b-f9d5-440d-8924-510c9fa8bf0d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:41.391663 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca95a5b-f9d5-440d-8924-510c9fa8bf0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:41.393023 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca95a5b-f9d5-440d-8924-510c9fa8bf0d None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:41.401306 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 597/1189] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:31:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:41.407071 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb848d-484e-4548-945f-a8d613fde154 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:41.496627 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:41.505416 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb848d-484e-4548-945f-a8d613fde154 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:41.539045 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe7490-a9c6-4480-a665-d36e376f56ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:41.539676 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe7490-a9c6-4480-a665-d36e376f56ec None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:41.539954 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe7490-a9c6-4480-a665-d36e376f56ec None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:41.540225 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe7490-a9c6-4480-a665-d36e376f56ec None admin] PATH_INFO: `/v3/users/f4fb1d1852974392855fd2d72ad6e6ee` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:41.541340 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fe7490-a9c6-4480-a665-d36e376f56ec None admin] RBAC: Authorizing `identity:get_user(user_id=f4fb1d1852974392855fd2d72ad6e6ee)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:41.542755 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fe7490-a9c6-4480-a665-d36e376f56ec None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:41.544345 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 593/1190] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:31:41 2022] GET /identity/v3/users/f4fb1d1852974392855fd2d72ad6e6ee => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:41.550020 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca95a5b-f9d5-440d-8924-510c9fa8bf0d None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:41.649789 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca95a5b-f9d5-440d-8924-510c9fa8bf0d None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:41.677765 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:41.682914 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79191c5-df61-4983-9ac6-ae2731ab7ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:41.683566 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79191c5-df61-4983-9ac6-ae2731ab7ab2 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:41.683840 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79191c5-df61-4983-9ac6-ae2731ab7ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:41.684362 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79191c5-df61-4983-9ac6-ae2731ab7ab2 None admin] PATH_INFO: `/v3/projects/7258eea809404d33870bfbc834958c03` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:41.685460 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79191c5-df61-4983-9ac6-ae2731ab7ab2 None admin] RBAC: Authorizing `identity:get_project(project_id=7258eea809404d33870bfbc834958c03)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:41.686865 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79191c5-df61-4983-9ac6-ae2731ab7ab2 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:41.688510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 598/1191] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:31:41 2022] GET /identity/v3/projects/7258eea809404d33870bfbc834958c03 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:41.694837 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe7490-a9c6-4480-a665-d36e376f56ec None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:41.790637 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe7490-a9c6-4480-a665-d36e376f56ec None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:41.823549 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9745f8-b747-4d7d-9c82-5b0a8fde4ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:41.824189 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9745f8-b747-4d7d-9c82-5b0a8fde4ba7 None admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:41.824466 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9745f8-b747-4d7d-9c82-5b0a8fde4ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:41.824736 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9745f8-b747-4d7d-9c82-5b0a8fde4ba7 None admin] PATH_INFO: `/v3/projects/7258eea809404d33870bfbc834958c03/users/f4fb1d1852974392855fd2d72ad6e6ee/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:41.826616 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9745f8-b747-4d7d-9c82-5b0a8fde4ba7 None admin] RBAC: Authorizing `identity:create_grant(project_id=7258eea809404d33870bfbc834958c03, user_id=f4fb1d1852974392855fd2d72ad6e6ee, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:41.828046 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9745f8-b747-4d7d-9c82-5b0a8fde4ba7 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:41.840720 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 594/1192] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 30 14:31:41 2022] PUT /identity/v3/projects/7258eea809404d33870bfbc834958c03/users/f4fb1d1852974392855fd2d72ad6e6ee/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:31:43.704083 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb19673-ba79-4787-b367-def84d0567e6 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:43.704649 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb19673-ba79-4787-b367-def84d0567e6 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:43.704649 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb19673-ba79-4787-b367-def84d0567e6 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:43.705965 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 599/1193] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:31:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:31:43.711125 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c375a-23de-45e3-b961-2e39af021c33 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:43.711557 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c375a-23de-45e3-b961-2e39af021c33 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:43.711718 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c375a-23de-45e3-b961-2e39af021c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:43.729426 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-141c375a-23de-45e3-b961-2e39af021c33 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:31:43.730605 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-141c375a-23de-45e3-b961-2e39af021c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:31:43.744746 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:43.796099 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 595/1194] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:31:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:31:44.064520 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb19673-ba79-4787-b367-def84d0567e6 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:44.163810 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb19673-ba79-4787-b367-def84d0567e6 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:44.196413 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87bba71-9d7d-44fa-abc0-f4988b85be6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:44.197024 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87bba71-9d7d-44fa-abc0-f4988b85be6f None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:44.197301 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87bba71-9d7d-44fa-abc0-f4988b85be6f None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:44.197572 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87bba71-9d7d-44fa-abc0-f4988b85be6f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:44.207813 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87bba71-9d7d-44fa-abc0-f4988b85be6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:31:44.212620 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-e87bba71-9d7d-44fa-abc0-f4988b85be6f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:31:44.212620 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:31:44.212620 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:31:44.212620 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:31:44.212620 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:31:44.212620 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:31:44.212620 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:31:44.212620 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:31:44.212620 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:31:44.212620 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:31:44.212620 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:31:44.212620 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:31:44.212620 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:31:44.212620 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:31:44.212620 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:31:44.212620 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:31:44.213632 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:31:44.213632 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:31:44.213632 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:31:44.213632 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:31:44.213632 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:31:44.213632 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:31:44.213632 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:31:44.213632 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:31:44.213632 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:31:44.213632 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:31:44.213632 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:31:44.213632 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:31:44.213632 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:31:44.213632 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:31:44.213632 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:31:44.213632 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:31:44.214486 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:31:44.214486 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:31:44.214486 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:31:44.214486 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:31:44.214486 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:31:44.214486 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:31:44.214486 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:31:44.214486 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:31:44.214486 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:31:44.214486 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:31:44.214486 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:31:44.214486 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:31:44.214486 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 600/1195] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 30 14:31:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:31:44.219694 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c375a-23de-45e3-b961-2e39af021c33 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:44.317966 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c375a-23de-45e3-b961-2e39af021c33 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:44.353915 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951f0d97-fb9b-4a7b-b72f-f9cacd74989b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:44.354515 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f0d97-fb9b-4a7b-b72f-f9cacd74989b None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:44.354792 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f0d97-fb9b-4a7b-b72f-f9cacd74989b None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:44.355061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f0d97-fb9b-4a7b-b72f-f9cacd74989b None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:44.355749 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951f0d97-fb9b-4a7b-b72f-f9cacd74989b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:44.356044 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951f0d97-fb9b-4a7b-b72f-f9cacd74989b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:44.357399 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951f0d97-fb9b-4a7b-b72f-f9cacd74989b None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:44.365143 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 596/1196] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:31:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:44.371145 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87bba71-9d7d-44fa-abc0-f4988b85be6f None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:44.429771 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:44.470160 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87bba71-9d7d-44fa-abc0-f4988b85be6f None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:44.508712 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ccf9b-9eb1-4814-9f3f-18ec0ece6692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:44.509332 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ccf9b-9eb1-4814-9f3f-18ec0ece6692 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:44.509614 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ccf9b-9eb1-4814-9f3f-18ec0ece6692 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:44.509890 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ccf9b-9eb1-4814-9f3f-18ec0ece6692 None admin] PATH_INFO: `/v3/users/f4fb1d1852974392855fd2d72ad6e6ee` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:44.511016 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48ccf9b-9eb1-4814-9f3f-18ec0ece6692 None admin] RBAC: Authorizing `identity:get_user(user_id=f4fb1d1852974392855fd2d72ad6e6ee)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:44.512477 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48ccf9b-9eb1-4814-9f3f-18ec0ece6692 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:44.514079 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 601/1197] 10.222.0.30 () {56 vars in 1177 bytes} [Tue Aug 30 14:31:44 2022] GET /identity/v3/users/f4fb1d1852974392855fd2d72ad6e6ee => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:44.519884 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951f0d97-fb9b-4a7b-b72f-f9cacd74989b None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:44.535668 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:44.619974 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951f0d97-fb9b-4a7b-b72f-f9cacd74989b None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:44.652244 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af7710-8a06-4189-8d24-3555152e75ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:44.652833 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af7710-8a06-4189-8d24-3555152e75ba None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:44.653107 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af7710-8a06-4189-8d24-3555152e75ba None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:44.653373 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af7710-8a06-4189-8d24-3555152e75ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:44.718750 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2af7710-8a06-4189-8d24-3555152e75ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:44.720197 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2af7710-8a06-4189-8d24-3555152e75ba None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:44.733296 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:44.815083 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 597/1198] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:31:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:31:44.821697 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ccf9b-9eb1-4814-9f3f-18ec0ece6692 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:44.919353 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ccf9b-9eb1-4814-9f3f-18ec0ece6692 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:44.945014 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:44.952680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36784a2-3450-4e49-a777-80f921bcce07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:44.953300 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36784a2-3450-4e49-a777-80f921bcce07 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:44.953578 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36784a2-3450-4e49-a777-80f921bcce07 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:44.953851 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36784a2-3450-4e49-a777-80f921bcce07 None admin] PATH_INFO: `/v3/projects/7258eea809404d33870bfbc834958c03` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:44.954957 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36784a2-3450-4e49-a777-80f921bcce07 None admin] RBAC: Authorizing `identity:get_project(project_id=7258eea809404d33870bfbc834958c03)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:44.956410 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36784a2-3450-4e49-a777-80f921bcce07 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:44.958031 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 602/1199] 10.222.0.30 () {56 vars in 1186 bytes} [Tue Aug 30 14:31:44 2022] GET /identity/v3/projects/7258eea809404d33870bfbc834958c03 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:44.964736 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af7710-8a06-4189-8d24-3555152e75ba None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:31:45.060099 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af7710-8a06-4189-8d24-3555152e75ba None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:31:45.073228 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:45.094490 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6af117-9e86-4817-8b4e-c965f74d9641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:31:45.095085 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6af117-9e86-4817-8b4e-c965f74d9641 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:31:45.095400 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6af117-9e86-4817-8b4e-c965f74d9641 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:31:45.095670 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6af117-9e86-4817-8b4e-c965f74d9641 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:31:45.096385 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6af117-9e86-4817-8b4e-c965f74d9641 None admin] RBAC: Adding query filter params (role.id=3cda0615d87d4af0860a2a5e9b4e2c05, scope.project.id=7258eea809404d33870bfbc834958c03, user.id=f4fb1d1852974392855fd2d72ad6e6ee) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:31:45.096686 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6af117-9e86-4817-8b4e-c965f74d9641 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:31:45.098096 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6af117-9e86-4817-8b4e-c965f74d9641 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:31:45.103898 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 598/1200] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:31:44 2022] GET /identity/v3/role_assignments?user.id=f4fb1d1852974392855fd2d72ad6e6ee&scope.project.id=7258eea809404d33870bfbc834958c03&role.id=3cda0615d87d4af0860a2a5e9b4e2c05 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:31:54.948888 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=2 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:31:55.101158 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=12 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:40.573500 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b0d297-c320-45f8-afaf-fd084d0d4150 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:40.573500 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b0d297-c320-45f8-afaf-fd084d0d4150 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:40.573500 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b0d297-c320-45f8-afaf-fd084d0d4150 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:40.573500 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 603/1201] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:32:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:32:40.582264 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c6465-33fe-4665-bc9a-6c1ab752c734 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:40.583268 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c6465-33fe-4665-bc9a-6c1ab752c734 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:40.583365 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c6465-33fe-4665-bc9a-6c1ab752c734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:40.605962 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-c90c6465-33fe-4665-bc9a-6c1ab752c734 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:32:40.607712 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-c90c6465-33fe-4665-bc9a-6c1ab752c734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:32:40.695198 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 599/1202] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:32:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:32:41.058490 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b0d297-c320-45f8-afaf-fd084d0d4150 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:41.200521 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b0d297-c320-45f8-afaf-fd084d0d4150 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:41.256702 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec1957-0b20-4175-bed8-cfbad22585de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:41.257368 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec1957-0b20-4175-bed8-cfbad22585de None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:41.257653 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec1957-0b20-4175-bed8-cfbad22585de None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:41.257928 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec1957-0b20-4175-bed8-cfbad22585de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:41.272861 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ec1957-0b20-4175-bed8-cfbad22585de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:41.274422 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ec1957-0b20-4175-bed8-cfbad22585de None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:41.291717 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-06ec1957-0b20-4175-bed8-cfbad22585de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:32:41.291717 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:41.291717 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:41.291717 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:41.291717 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:41.291717 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:41.291717 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:41.291717 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:41.291717 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:41.291717 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:41.291717 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:41.291717 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:41.291717 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:32:41.291717 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:32:41.291717 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:32:41.291717 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:32:41.293024 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:41.293024 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:41.293024 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:41.293024 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:41.293024 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:41.293024 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:41.293024 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:41.293024 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:41.293024 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:41.293024 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:41.293024 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:41.293024 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:41.293024 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:41.293024 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:41.293024 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:41.293024 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:41.293024 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:32:41.294241 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:32:41.294241 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:32:41.294241 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:32:41.295751 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 604/1203] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:32:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:32:41.302062 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c6465-33fe-4665-bc9a-6c1ab752c734 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:41.432460 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c6465-33fe-4665-bc9a-6c1ab752c734 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:41.476566 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:41.484793 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dfa822-98b1-48c7-a5f1-90efdd384bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:41.485454 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dfa822-98b1-48c7-a5f1-90efdd384bb7 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:41.485771 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dfa822-98b1-48c7-a5f1-90efdd384bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:41.486044 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dfa822-98b1-48c7-a5f1-90efdd384bb7 None admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:41.486715 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dfa822-98b1-48c7-a5f1-90efdd384bb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:41.487018 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dfa822-98b1-48c7-a5f1-90efdd384bb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:41.488484 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dfa822-98b1-48c7-a5f1-90efdd384bb7 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:41.502075 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 600/1204] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:32:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:41.509176 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec1957-0b20-4175-bed8-cfbad22585de None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:41.574374 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:41.628297 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec1957-0b20-4175-bed8-cfbad22585de None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:41.669579 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824e4fdf-eab9-4259-9612-9ce5d3dafd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:41.670192 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e4fdf-eab9-4259-9612-9ce5d3dafd37 None admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:41.670569 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e4fdf-eab9-4259-9612-9ce5d3dafd37 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:41.670826 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e4fdf-eab9-4259-9612-9ce5d3dafd37 None admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:41.671512 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824e4fdf-eab9-4259-9612-9ce5d3dafd37 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:41.672922 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824e4fdf-eab9-4259-9612-9ce5d3dafd37 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:41.697080 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 605/1205] 10.222.0.30 () {60 vars in 1132 bytes} [Tue Aug 30 14:32:41 2022] POST /identity/v3/users => generated 251 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:32:43.789741 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d052a-b7ad-44e3-94b2-0d35648af504 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:43.790294 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d052a-b7ad-44e3-94b2-0d35648af504 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:43.790294 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d052a-b7ad-44e3-94b2-0d35648af504 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:43.791835 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 601/1206] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:32:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:32:43.797579 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8f72c-0357-4e17-b99c-f4817aae93ee None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:43.798112 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8f72c-0357-4e17-b99c-f4817aae93ee None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:43.798345 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8f72c-0357-4e17-b99c-f4817aae93ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:43.822897 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-21f8f72c-0357-4e17-b99c-f4817aae93ee None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:32:43.824162 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-21f8f72c-0357-4e17-b99c-f4817aae93ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:32:43.922226 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 606/1207] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:32:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 14:32:44.249327 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d052a-b7ad-44e3-94b2-0d35648af504 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:44.382220 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d052a-b7ad-44e3-94b2-0d35648af504 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:44.422225 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db76a644-316e-407a-8d93-853a6e2ce554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:44.422898 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76a644-316e-407a-8d93-853a6e2ce554 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:44.423154 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76a644-316e-407a-8d93-853a6e2ce554 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:44.423469 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76a644-316e-407a-8d93-853a6e2ce554 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:44.436255 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db76a644-316e-407a-8d93-853a6e2ce554 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:32:44.441028 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-db76a644-316e-407a-8d93-853a6e2ce554 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:32:44.441028 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:44.441028 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:44.441028 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:44.441028 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:44.441028 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:44.441028 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:44.441028 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:44.441028 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:44.441028 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:44.441028 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:44.441028 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:44.441028 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:32:44.441028 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:32:44.441028 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:32:44.441028 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:32:44.442651 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:32:44.442651 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:32:44.442651 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:32:44.442651 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:32:44.442651 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:44.442651 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:44.442651 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:44.442651 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:44.442651 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:44.442651 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:44.442651 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:44.442651 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:44.442651 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:44.442651 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:44.442651 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:44.442651 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:44.443931 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:44.443931 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:44.443931 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:44.443931 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:44.443931 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:32:44.443931 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:32:44.443931 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:32:44.443931 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:32:44.443931 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:32:44.443931 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:32:44.443931 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:32:44.443931 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:32:44.443931 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 602/1208] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 30 14:32:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:32:44.449819 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f8f72c-0357-4e17-b99c-f4817aae93ee None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:44.500447 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:44.581295 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f8f72c-0357-4e17-b99c-f4817aae93ee None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:44.622439 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30aedd5-f141-483a-acca-b3e7b232a8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:44.623061 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30aedd5-f141-483a-acca-b3e7b232a8ce None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:44.623374 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30aedd5-f141-483a-acca-b3e7b232a8ce None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:44.623645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30aedd5-f141-483a-acca-b3e7b232a8ce None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:44.624315 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30aedd5-f141-483a-acca-b3e7b232a8ce None admin] RBAC: Adding query filter params (name=service) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:44.624615 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30aedd5-f141-483a-acca-b3e7b232a8ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:44.625991 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30aedd5-f141-483a-acca-b3e7b232a8ce None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:44.635575 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 607/1209] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 30 14:32:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:44.642095 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db76a644-316e-407a-8d93-853a6e2ce554 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:44.694920 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:44.757742 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db76a644-316e-407a-8d93-853a6e2ce554 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:44.800817 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9465ebd8-f735-43d8-bb44-709e92c17303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:44.802071 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9465ebd8-f735-43d8-bb44-709e92c17303 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:44.802316 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9465ebd8-f735-43d8-bb44-709e92c17303 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:44.802585 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9465ebd8-f735-43d8-bb44-709e92c17303 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:44.815174 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9465ebd8-f735-43d8-bb44-709e92c17303 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:44.816688 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9465ebd8-f735-43d8-bb44-709e92c17303 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:44.835556 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-9465ebd8-f735-43d8-bb44-709e92c17303 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:32:44.835556 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:44.835556 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:44.835556 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:44.835556 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:44.835556 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:44.835556 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:44.835556 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:44.835556 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:44.835556 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:44.835556 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:44.835556 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:44.835556 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:32:44.835556 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:32:44.835556 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:32:44.835556 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:32:44.836650 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:44.836650 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:44.836650 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:44.836650 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:44.836650 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:44.836650 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:44.836650 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:44.836650 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:44.836650 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:44.836650 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:44.836650 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:44.836650 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:44.836650 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:44.836650 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:44.836650 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:44.836650 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:44.836650 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:32:44.842619 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:32:44.842619 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:32:44.842619 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:32:44.842619 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 603/1210] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:32:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:32:44.845003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30aedd5-f141-483a-acca-b3e7b232a8ce None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:44.979632 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:44.982902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30aedd5-f141-483a-acca-b3e7b232a8ce None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:45.029679 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412891f1-4d91-4b2b-a03f-62cbd604cedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:45.030248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412891f1-4d91-4b2b-a03f-62cbd604cedf None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:45.030577 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412891f1-4d91-4b2b-a03f-62cbd604cedf None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:45.030775 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412891f1-4d91-4b2b-a03f-62cbd604cedf None admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:45.031522 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412891f1-4d91-4b2b-a03f-62cbd604cedf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:45.031857 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412891f1-4d91-4b2b-a03f-62cbd604cedf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:45.033300 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412891f1-4d91-4b2b-a03f-62cbd604cedf None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:45.046753 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 608/1211] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:32:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:45.054053 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9465ebd8-f735-43d8-bb44-709e92c17303 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:45.169840 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9465ebd8-f735-43d8-bb44-709e92c17303 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:45.171923 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:45.212355 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6cd743-52f2-4dd6-8d20-1e88b7cafd4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:45.213026 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6cd743-52f2-4dd6-8d20-1e88b7cafd4b None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:45.213311 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6cd743-52f2-4dd6-8d20-1e88b7cafd4b None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:45.213586 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6cd743-52f2-4dd6-8d20-1e88b7cafd4b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:45.222199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6cd743-52f2-4dd6-8d20-1e88b7cafd4b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:45.224356 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6cd743-52f2-4dd6-8d20-1e88b7cafd4b None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:45.237496 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-9b6cd743-52f2-4dd6-8d20-1e88b7cafd4b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 14:32:45.237496 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:45.237496 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:45.237496 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:45.237496 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:45.237496 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:45.237496 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:45.237496 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:45.237496 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:45.237496 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:45.237496 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:45.237496 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:45.237496 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:32:45.237496 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:32:45.237496 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:32:45.237496 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:32:45.239484 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:45.239484 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:45.239484 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:32:45.239484 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:32:45.239484 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:32:45.239484 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:32:45.239484 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:45.239484 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:45.239484 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:45.239484 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:45.239484 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:45.239484 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:45.239484 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:45.239484 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:45.239484 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:45.239484 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:45.239484 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:45.239484 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:45.240503 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:45.240503 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:45.240503 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:32:45.240503 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:32:45.240503 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:32:45.240503 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:32:45.240503 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:32:45.240503 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:32:45.240503 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 14:32:45.240503 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:32:45.242814 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 604/1212] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:32:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:32:45.248806 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412891f1-4d91-4b2b-a03f-62cbd604cedf None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:45.369603 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412891f1-4d91-4b2b-a03f-62cbd604cedf None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:45.415812 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d2a65c-bb11-4c1d-99e5-77482bdebec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:45.416474 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d2a65c-bb11-4c1d-99e5-77482bdebec1 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:45.416779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d2a65c-bb11-4c1d-99e5-77482bdebec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:45.417244 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d2a65c-bb11-4c1d-99e5-77482bdebec1 None admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:45.418922 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d2a65c-bb11-4c1d-99e5-77482bdebec1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:45.419574 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d2a65c-bb11-4c1d-99e5-77482bdebec1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:45.421640 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d2a65c-bb11-4c1d-99e5-77482bdebec1 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:45.446827 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 609/1213] 10.222.0.30 () {56 vars in 1143 bytes} [Tue Aug 30 14:32:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:45.453923 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6cd743-52f2-4dd6-8d20-1e88b7cafd4b None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:45.572921 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6cd743-52f2-4dd6-8d20-1e88b7cafd4b None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:45.609706 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2509d1-bb8e-44b1-8ff8-e03a26ae669d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:45.611183 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2509d1-bb8e-44b1-8ff8-e03a26ae669d None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:45.611183 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2509d1-bb8e-44b1-8ff8-e03a26ae669d None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:45.611183 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2509d1-bb8e-44b1-8ff8-e03a26ae669d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:45.647414 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:45.690106 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2509d1-bb8e-44b1-8ff8-e03a26ae669d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:45.691616 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2509d1-bb8e-44b1-8ff8-e03a26ae669d None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:45.809672 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 605/1214] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:32:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:32:45.815997 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d2a65c-bb11-4c1d-99e5-77482bdebec1 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:45.842118 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:45.964542 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d2a65c-bb11-4c1d-99e5-77482bdebec1 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:45.996236 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ddcf1-b2b8-402d-a937-64bc2d2a8763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:45.996869 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ddcf1-b2b8-402d-a937-64bc2d2a8763 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:45.997175 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ddcf1-b2b8-402d-a937-64bc2d2a8763 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:45.997413 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ddcf1-b2b8-402d-a937-64bc2d2a8763 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:46.008691 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97ddcf1-b2b8-402d-a937-64bc2d2a8763 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:46.010199 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97ddcf1-b2b8-402d-a937-64bc2d2a8763 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:46.025667 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-d97ddcf1-b2b8-402d-a937-64bc2d2a8763 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:32:46.025667 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:46.025667 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:46.025667 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:46.025667 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:46.025667 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:46.025667 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:46.025667 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:46.025667 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:46.025667 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:46.025667 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:46.025667 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:46.025667 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:32:46.025667 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:32:46.025667 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:32:46.025667 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:32:46.026649 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:46.026649 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:46.026649 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:46.026649 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:46.026649 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:46.026649 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:46.026649 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:46.026649 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:46.026649 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:46.026649 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:46.026649 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:46.026649 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:46.026649 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:46.026649 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:46.026649 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:46.026649 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:46.026649 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:32:46.034390 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:32:46.034390 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:32:46.034390 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:32:46.034390 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 610/1215] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:32:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:32:46.034671 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2509d1-bb8e-44b1-8ff8-e03a26ae669d None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:46.095749 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:46.137595 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2509d1-bb8e-44b1-8ff8-e03a26ae669d None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:46.173567 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9cd338-fa20-42d9-ba4c-7684a7c17efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:46.174218 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9cd338-fa20-42d9-ba4c-7684a7c17efb None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:46.174498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9cd338-fa20-42d9-ba4c-7684a7c17efb None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:46.174794 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9cd338-fa20-42d9-ba4c-7684a7c17efb None admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:46.175460 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9cd338-fa20-42d9-ba4c-7684a7c17efb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:46.175757 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9cd338-fa20-42d9-ba4c-7684a7c17efb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:46.177377 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9cd338-fa20-42d9-ba4c-7684a7c17efb None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:46.189488 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 606/1216] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:32:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:46.195650 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ddcf1-b2b8-402d-a937-64bc2d2a8763 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:46.274429 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:46.310305 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ddcf1-b2b8-402d-a937-64bc2d2a8763 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:46.350435 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f27a49a-6e8f-4d37-ae90-5138f0cc86e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:46.351074 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f27a49a-6e8f-4d37-ae90-5138f0cc86e1 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:46.351397 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f27a49a-6e8f-4d37-ae90-5138f0cc86e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:46.351674 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f27a49a-6e8f-4d37-ae90-5138f0cc86e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:46.358720 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f27a49a-6e8f-4d37-ae90-5138f0cc86e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:46.360304 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f27a49a-6e8f-4d37-ae90-5138f0cc86e1 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:46.372034 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-4f27a49a-6e8f-4d37-ae90-5138f0cc86e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:32:46.372034 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:46.372034 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:46.372034 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:46.372034 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:46.372034 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:46.372034 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:46.372034 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:46.372034 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:46.372034 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:46.372034 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:46.372034 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:46.372034 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:32:46.372034 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:32:46.372034 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:32:46.374106 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:32:46.374106 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:46.374106 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:46.374106 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:46.374106 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:46.374106 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:46.374106 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:46.374106 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:46.374106 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:46.374106 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:46.374106 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:46.374106 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:46.374106 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:46.374106 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:46.374106 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:46.374106 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:46.374106 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:46.375616 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:32:46.375616 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:32:46.375616 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:32:46.375616 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:32:46.375616 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:32:46.375616 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:32:46.375616 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:32:46.375616 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:32:46.375616 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 611/1217] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:32:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:32:46.381827 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9cd338-fa20-42d9-ba4c-7684a7c17efb None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:46.495332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9cd338-fa20-42d9-ba4c-7684a7c17efb None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:46.523111 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:46.541309 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa21c85-932b-402a-96e4-4b02df53991e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:46.542212 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa21c85-932b-402a-96e4-4b02df53991e None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:46.542724 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa21c85-932b-402a-96e4-4b02df53991e None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:46.543202 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa21c85-932b-402a-96e4-4b02df53991e None admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:46.544183 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa21c85-932b-402a-96e4-4b02df53991e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:46.544489 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa21c85-932b-402a-96e4-4b02df53991e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:46.545946 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa21c85-932b-402a-96e4-4b02df53991e None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:46.565923 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 607/1218] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 30 14:32:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:46.568175 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f27a49a-6e8f-4d37-ae90-5138f0cc86e1 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:46.687003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f27a49a-6e8f-4d37-ae90-5138f0cc86e1 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:46.718080 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:46.725120 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91445ba2-30ab-4d90-bf93-a7c201a76b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:46.727128 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91445ba2-30ab-4d90-bf93-a7c201a76b28 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:46.727648 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91445ba2-30ab-4d90-bf93-a7c201a76b28 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:46.727925 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91445ba2-30ab-4d90-bf93-a7c201a76b28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:46.728731 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91445ba2-30ab-4d90-bf93-a7c201a76b28 None admin] RBAC: Adding query filter params (role.id=1316db16eac54629a283545f1601a236, scope.project.id=01cf71ca5abf4576b89d10c85af6d2de, user.id=dcf302a170934c25b39699088dac326a) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:46.729101 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91445ba2-30ab-4d90-bf93-a7c201a76b28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:46.730860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91445ba2-30ab-4d90-bf93-a7c201a76b28 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:46.737318 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 612/1219] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:32:46 2022] GET /identity/v3/role_assignments?user.id=dcf302a170934c25b39699088dac326a&scope.project.id=01cf71ca5abf4576b89d10c85af6d2de&role.id=1316db16eac54629a283545f1601a236 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:48.788207 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce2529b-fbe0-468e-a426-3431522b90cd None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:48.788789 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce2529b-fbe0-468e-a426-3431522b90cd None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:48.788892 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce2529b-fbe0-468e-a426-3431522b90cd None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:48.790359 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 608/1220] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:32:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:32:48.796046 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef831c-3ba3-4043-908c-3189f33a54a2 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:48.796526 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef831c-3ba3-4043-908c-3189f33a54a2 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:48.796629 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef831c-3ba3-4043-908c-3189f33a54a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:48.817222 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-dbef831c-3ba3-4043-908c-3189f33a54a2 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:32:48.818732 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-dbef831c-3ba3-4043-908c-3189f33a54a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:32:48.890779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 613/1221] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:32:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:32:49.181947 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce2529b-fbe0-468e-a426-3431522b90cd None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:49.294726 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce2529b-fbe0-468e-a426-3431522b90cd None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:49.339466 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa942a9-8679-42e3-8346-ca2b120c18e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:49.340097 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa942a9-8679-42e3-8346-ca2b120c18e6 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:49.340375 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa942a9-8679-42e3-8346-ca2b120c18e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:49.340648 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa942a9-8679-42e3-8346-ca2b120c18e6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:49.351613 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa942a9-8679-42e3-8346-ca2b120c18e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:32:49.356339 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-4fa942a9-8679-42e3-8346-ca2b120c18e6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:32:49.356339 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:49.356339 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:49.356339 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:49.356339 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:49.356339 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:49.356339 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:49.356339 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:49.356339 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:49.356339 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:49.356339 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:49.356339 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:49.356339 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:32:49.356339 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:32:49.356339 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:32:49.356339 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:32:49.357497 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:32:49.357497 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:32:49.357497 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:32:49.357497 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:32:49.357497 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:49.357497 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:49.357497 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:49.357497 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:49.357497 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:49.357497 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:49.357497 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:49.357497 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:49.357497 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:49.357497 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:49.357497 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:49.357497 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:49.358436 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:49.358436 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:49.358436 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:49.358436 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:49.358436 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:32:49.358436 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:32:49.358436 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:32:49.358436 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:32:49.358436 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:32:49.358436 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:32:49.358436 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:32:49.358436 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:32:49.358436 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 609/1222] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 30 14:32:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:32:49.364224 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbef831c-3ba3-4043-908c-3189f33a54a2 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:49.482163 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbef831c-3ba3-4043-908c-3189f33a54a2 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:49.511715 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:49.523642 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10dc6b6-e438-4e19-b49d-820f3ee62762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:49.524259 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10dc6b6-e438-4e19-b49d-820f3ee62762 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:49.524530 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10dc6b6-e438-4e19-b49d-820f3ee62762 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:49.524803 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10dc6b6-e438-4e19-b49d-820f3ee62762 None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:49.525562 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10dc6b6-e438-4e19-b49d-820f3ee62762 None admin] RBAC: Adding query filter params (name=service) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:49.525903 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10dc6b6-e438-4e19-b49d-820f3ee62762 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:49.527429 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10dc6b6-e438-4e19-b49d-820f3ee62762 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:49.535691 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 614/1223] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 30 14:32:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:49.541488 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa942a9-8679-42e3-8346-ca2b120c18e6 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:49.587180 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:49.647912 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa942a9-8679-42e3-8346-ca2b120c18e6 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:49.685585 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e3a277-d61c-42eb-ad76-17bf4ae6b789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:49.686215 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e3a277-d61c-42eb-ad76-17bf4ae6b789 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:49.686499 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e3a277-d61c-42eb-ad76-17bf4ae6b789 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:49.686789 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e3a277-d61c-42eb-ad76-17bf4ae6b789 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:49.699899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e3a277-d61c-42eb-ad76-17bf4ae6b789 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:49.701591 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e3a277-d61c-42eb-ad76-17bf4ae6b789 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:49.719515 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-d3e3a277-d61c-42eb-ad76-17bf4ae6b789 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:32:49.719515 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:49.719515 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:49.719515 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:49.719515 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:49.719515 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:49.719515 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:49.719515 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:49.719515 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:49.719515 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:49.719515 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:49.719515 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:49.719515 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:32:49.719515 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:32:49.719515 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:32:49.719515 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:32:49.720654 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:49.720654 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:49.720654 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:49.720654 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:49.720654 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:49.720654 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:49.720654 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:49.720654 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:49.720654 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:49.720654 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:49.720654 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:49.720654 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:49.720654 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:49.720654 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:49.720654 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:49.720654 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:49.720654 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:32:49.721709 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:32:49.721709 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:32:49.721709 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:32:49.727290 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 610/1224] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:32:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:32:49.728555 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10dc6b6-e438-4e19-b49d-820f3ee62762 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:49.840648 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10dc6b6-e438-4e19-b49d-820f3ee62762 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:49.878151 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af97edaa-9998-4186-b441-f5b5890b5947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:49.878789 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97edaa-9998-4186-b441-f5b5890b5947 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:49.879034 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97edaa-9998-4186-b441-f5b5890b5947 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:49.879903 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97edaa-9998-4186-b441-f5b5890b5947 None admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:49.880670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af97edaa-9998-4186-b441-f5b5890b5947 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:49.881005 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af97edaa-9998-4186-b441-f5b5890b5947 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:49.882466 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af97edaa-9998-4186-b441-f5b5890b5947 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:49.895628 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 615/1225] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:32:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:49.901798 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e3a277-d61c-42eb-ad76-17bf4ae6b789 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:50.012697 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:50.020383 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e3a277-d61c-42eb-ad76-17bf4ae6b789 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:50.059186 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d2e533-55b2-48bd-85ba-eb1da9ece512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:50.059975 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2e533-55b2-48bd-85ba-eb1da9ece512 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:50.060239 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2e533-55b2-48bd-85ba-eb1da9ece512 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:50.060513 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2e533-55b2-48bd-85ba-eb1da9ece512 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:50.068504 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d2e533-55b2-48bd-85ba-eb1da9ece512 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:50.069987 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d2e533-55b2-48bd-85ba-eb1da9ece512 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:50.081415 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-c0d2e533-55b2-48bd-85ba-eb1da9ece512 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 14:32:50.081415 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:50.081415 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:50.081415 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:50.081415 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:50.081415 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:50.081415 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:50.081415 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:50.081415 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:50.081415 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:50.081415 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:50.081415 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:50.081415 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:32:50.081415 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:32:50.081415 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:32:50.081415 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:32:50.082624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:50.082624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:50.082624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:32:50.082624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:32:50.082624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:32:50.082624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:32:50.082624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:50.082624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:50.082624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:50.082624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:50.082624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:50.082624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:50.082624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:50.082624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:50.082624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:50.082624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:50.082624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:50.082624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:50.083665 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:50.083665 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:50.083665 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:32:50.083665 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:32:50.083665 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:32:50.083665 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:32:50.083665 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:32:50.083665 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:32:50.083665 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 14:32:50.083665 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:32:50.083665 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 611/1226] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:32:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:32:50.088585 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af97edaa-9998-4186-b441-f5b5890b5947 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:50.141391 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:50.200291 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af97edaa-9998-4186-b441-f5b5890b5947 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:50.237692 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df41ac-8e39-4067-8865-1993095061b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:50.238334 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df41ac-8e39-4067-8865-1993095061b3 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:50.239172 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df41ac-8e39-4067-8865-1993095061b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:50.239543 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df41ac-8e39-4067-8865-1993095061b3 None admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:50.240374 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df41ac-8e39-4067-8865-1993095061b3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:50.240714 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df41ac-8e39-4067-8865-1993095061b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:50.242376 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df41ac-8e39-4067-8865-1993095061b3 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:50.261414 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 616/1227] 10.222.0.30 () {56 vars in 1143 bytes} [Tue Aug 30 14:32:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:50.267071 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d2e533-55b2-48bd-85ba-eb1da9ece512 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:50.387965 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d2e533-55b2-48bd-85ba-eb1da9ece512 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:50.426320 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e991f19-3e63-49c8-ac92-c5526bb0ec08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:50.426982 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e991f19-3e63-49c8-ac92-c5526bb0ec08 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:50.427357 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e991f19-3e63-49c8-ac92-c5526bb0ec08 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:50.427632 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e991f19-3e63-49c8-ac92-c5526bb0ec08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:50.438509 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e991f19-3e63-49c8-ac92-c5526bb0ec08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:50.439979 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e991f19-3e63-49c8-ac92-c5526bb0ec08 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:50.457700 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-8e991f19-3e63-49c8-ac92-c5526bb0ec08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:32:50.457700 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:50.457700 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:50.457700 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:50.457700 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:50.457700 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:50.457700 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:50.457700 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:50.457700 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:50.457700 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:50.457700 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:50.457700 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:50.457700 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:32:50.457700 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:32:50.457700 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:32:50.457700 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:32:50.458963 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:50.458963 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:50.458963 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:50.458963 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:50.458963 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:50.458963 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:50.458963 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:50.458963 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:50.458963 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:50.458963 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:50.458963 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:50.458963 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:50.458963 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:50.458963 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:50.458963 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:50.458963 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:50.458963 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:32:50.460256 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:32:50.460256 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:32:50.460256 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:32:50.460256 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 612/1228] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:32:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:32:50.465705 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df41ac-8e39-4067-8865-1993095061b3 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:50.571433 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df41ac-8e39-4067-8865-1993095061b3 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:50.573894 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:50.608099 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2bd53d-ea91-45e3-8c30-fc6ccb72264a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:50.608707 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2bd53d-ea91-45e3-8c30-fc6ccb72264a None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:50.609037 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2bd53d-ea91-45e3-8c30-fc6ccb72264a None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:50.609263 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2bd53d-ea91-45e3-8c30-fc6ccb72264a None admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:50.609904 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2bd53d-ea91-45e3-8c30-fc6ccb72264a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:50.610208 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2bd53d-ea91-45e3-8c30-fc6ccb72264a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:50.611636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2bd53d-ea91-45e3-8c30-fc6ccb72264a None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:50.623891 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 617/1229] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:32:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:50.630716 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e991f19-3e63-49c8-ac92-c5526bb0ec08 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:50.641011 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:50.752861 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e991f19-3e63-49c8-ac92-c5526bb0ec08 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:50.790031 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a22acb-8349-4ed0-89f2-8f37ded98e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:50.790685 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a22acb-8349-4ed0-89f2-8f37ded98e86 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:50.790964 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a22acb-8349-4ed0-89f2-8f37ded98e86 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:50.791259 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a22acb-8349-4ed0-89f2-8f37ded98e86 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:50.802561 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a22acb-8349-4ed0-89f2-8f37ded98e86 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:50.804542 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a22acb-8349-4ed0-89f2-8f37ded98e86 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:50.819441 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-20a22acb-8349-4ed0-89f2-8f37ded98e86 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:32:50.819441 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:50.819441 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:50.819441 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:50.819441 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:50.819441 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:50.819441 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:50.819441 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:50.819441 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:50.819441 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:50.819441 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:50.819441 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:50.819441 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:32:50.819441 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:32:50.819441 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:32:50.820510 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:32:50.820510 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:50.820510 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:50.820510 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:50.820510 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:50.820510 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:50.820510 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:50.820510 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:50.820510 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:50.820510 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:50.820510 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:50.820510 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:50.820510 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:50.820510 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:50.820510 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:50.820510 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:50.820510 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:50.821438 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:32:50.821438 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:32:50.821438 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:32:50.821438 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:32:50.821438 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:32:50.821438 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:32:50.821438 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:32:50.821438 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:32:50.821438 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 613/1230] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:32:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:32:50.826771 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2bd53d-ea91-45e3-8c30-fc6ccb72264a None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:50.934054 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2bd53d-ea91-45e3-8c30-fc6ccb72264a None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:50.969212 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a568d9-7027-4221-9cf7-7d6cc1a548af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:50.969815 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a568d9-7027-4221-9cf7-7d6cc1a548af None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:50.970098 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a568d9-7027-4221-9cf7-7d6cc1a548af None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:50.970376 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a568d9-7027-4221-9cf7-7d6cc1a548af None admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:50.971108 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a568d9-7027-4221-9cf7-7d6cc1a548af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:50.971444 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a568d9-7027-4221-9cf7-7d6cc1a548af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:50.973420 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a568d9-7027-4221-9cf7-7d6cc1a548af None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:50.986522 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 618/1231] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 30 14:32:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:50.993836 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a22acb-8349-4ed0-89f2-8f37ded98e86 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:51.069368 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:51.109353 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a22acb-8349-4ed0-89f2-8f37ded98e86 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:51.143576 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483e5f63-c091-4cee-911e-6413fff26428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:51.144242 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e5f63-c091-4cee-911e-6413fff26428 None admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:51.144517 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e5f63-c091-4cee-911e-6413fff26428 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:51.145322 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e5f63-c091-4cee-911e-6413fff26428 None admin] PATH_INFO: `/v3/projects/01cf71ca5abf4576b89d10c85af6d2de/users/dcf302a170934c25b39699088dac326a/roles/1316db16eac54629a283545f1601a236` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:51.162368 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483e5f63-c091-4cee-911e-6413fff26428 None admin] RBAC: Authorizing `identity:create_grant(project_id=01cf71ca5abf4576b89d10c85af6d2de, user_id=dcf302a170934c25b39699088dac326a, role_id=1316db16eac54629a283545f1601a236)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:51.164506 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483e5f63-c091-4cee-911e-6413fff26428 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:51.178936 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 614/1232] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 30 14:32:50 2022] PUT /identity/v3/projects/01cf71ca5abf4576b89d10c85af6d2de/users/dcf302a170934c25b39699088dac326a/roles/1316db16eac54629a283545f1601a236 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:32:53.195736 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5363aa-f58a-415e-a29c-32f0273a21a4 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:53.196283 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5363aa-f58a-415e-a29c-32f0273a21a4 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:53.196283 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5363aa-f58a-415e-a29c-32f0273a21a4 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:53.197643 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 619/1233] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:32:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:32:53.204099 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc5fe9-eeba-4ac5-93db-a94d40cdde30 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:53.204560 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc5fe9-eeba-4ac5-93db-a94d40cdde30 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:53.204672 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc5fe9-eeba-4ac5-93db-a94d40cdde30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:53.224379 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-50cc5fe9-eeba-4ac5-93db-a94d40cdde30 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:32:53.225707 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-50cc5fe9-eeba-4ac5-93db-a94d40cdde30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:32:53.300637 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 615/1234] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:32:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:32:53.600702 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5363aa-f58a-415e-a29c-32f0273a21a4 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:53.629694 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:53.718912 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5363aa-f58a-415e-a29c-32f0273a21a4 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:53.759766 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14161b-0aec-4663-9047-bec8ab58e167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:53.760470 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14161b-0aec-4663-9047-bec8ab58e167 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:53.760814 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14161b-0aec-4663-9047-bec8ab58e167 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:53.761101 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14161b-0aec-4663-9047-bec8ab58e167 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:53.772743 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c14161b-0aec-4663-9047-bec8ab58e167 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:32:53.777540 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-3c14161b-0aec-4663-9047-bec8ab58e167 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:32:53.777540 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:53.777540 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:53.777540 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:53.777540 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:53.777540 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:53.777540 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:53.777540 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:53.777540 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:53.777540 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:53.777540 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:53.777540 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:53.777540 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:32:53.777540 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:32:53.777540 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:32:53.777540 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:32:53.778723 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:32:53.778723 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:32:53.778723 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:32:53.778723 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:32:53.778723 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:53.778723 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:53.778723 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:53.778723 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:53.778723 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:53.778723 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:53.778723 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:53.778723 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:53.778723 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:53.778723 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:53.778723 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:53.778723 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:53.779777 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:53.779777 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:53.779777 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:53.779777 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:53.779777 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:32:53.779777 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:32:53.779777 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:32:53.779777 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:32:53.779777 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:32:53.779777 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:32:53.779777 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 14:32:53.779777 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:32:53.779777 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 620/1235] 10.222.0.30 () {56 vars in 1102 bytes} [Tue Aug 30 14:32:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:32:53.788262 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc5fe9-eeba-4ac5-93db-a94d40cdde30 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:53.840982 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:53.901400 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc5fe9-eeba-4ac5-93db-a94d40cdde30 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:53.941249 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12f60eb-aa7d-452f-ae84-fa64d2c89ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:53.941890 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f60eb-aa7d-452f-ae84-fa64d2c89ad2 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:53.942174 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f60eb-aa7d-452f-ae84-fa64d2c89ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:53.942434 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f60eb-aa7d-452f-ae84-fa64d2c89ad2 None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:53.943093 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12f60eb-aa7d-452f-ae84-fa64d2c89ad2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:53.943427 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12f60eb-aa7d-452f-ae84-fa64d2c89ad2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:53.944779 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12f60eb-aa7d-452f-ae84-fa64d2c89ad2 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:53.952925 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 616/1236] 10.222.0.30 () {56 vars in 1103 bytes} [Tue Aug 30 14:32:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:53.958819 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14161b-0aec-4663-9047-bec8ab58e167 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:54.053438 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:54.072299 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14161b-0aec-4663-9047-bec8ab58e167 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:54.112573 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e250c2e6-3905-448a-b52d-3dd1fa969c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:54.113366 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e250c2e6-3905-448a-b52d-3dd1fa969c24 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:54.113836 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e250c2e6-3905-448a-b52d-3dd1fa969c24 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:54.114131 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e250c2e6-3905-448a-b52d-3dd1fa969c24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:54.126445 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e250c2e6-3905-448a-b52d-3dd1fa969c24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:54.128060 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e250c2e6-3905-448a-b52d-3dd1fa969c24 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:54.146039 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-e250c2e6-3905-448a-b52d-3dd1fa969c24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:32:54.146039 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:54.146039 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:54.146039 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:54.146039 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:54.146039 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:54.146039 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:54.146039 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:54.146039 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:54.146039 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:54.146039 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:54.146039 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:54.146039 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:32:54.146039 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:32:54.146039 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:32:54.146039 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:32:54.147427 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:54.147427 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:54.147427 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:54.147427 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:54.147427 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:54.147427 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:54.147427 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:54.147427 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:54.147427 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:54.147427 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:54.147427 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:54.147427 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:54.147427 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:54.147427 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:54.147427 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:54.147427 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:54.147427 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:32:54.148524 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:32:54.148524 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:32:54.148524 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:32:54.148524 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 621/1237] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:32:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:32:54.153270 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12f60eb-aa7d-452f-ae84-fa64d2c89ad2 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:54.269423 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12f60eb-aa7d-452f-ae84-fa64d2c89ad2 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:54.280392 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:54.314172 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e8dd0c-df7c-430c-b8ff-3e0ffaa8c6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:54.314811 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e8dd0c-df7c-430c-b8ff-3e0ffaa8c6aa None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:54.315094 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e8dd0c-df7c-430c-b8ff-3e0ffaa8c6aa None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:54.315403 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e8dd0c-df7c-430c-b8ff-3e0ffaa8c6aa None admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:54.316050 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e8dd0c-df7c-430c-b8ff-3e0ffaa8c6aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:54.316344 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e8dd0c-df7c-430c-b8ff-3e0ffaa8c6aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:54.318093 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e8dd0c-df7c-430c-b8ff-3e0ffaa8c6aa None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:54.334039 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 617/1238] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:32:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:54.340825 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e250c2e6-3905-448a-b52d-3dd1fa969c24 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:54.454294 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e250c2e6-3905-448a-b52d-3dd1fa969c24 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:54.490885 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b754d69-dcb3-4174-807c-96e5c56465e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:54.491543 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b754d69-dcb3-4174-807c-96e5c56465e6 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:54.491831 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b754d69-dcb3-4174-807c-96e5c56465e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:54.492102 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b754d69-dcb3-4174-807c-96e5c56465e6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:54.497789 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:54.503498 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b754d69-dcb3-4174-807c-96e5c56465e6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:54.505201 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b754d69-dcb3-4174-807c-96e5c56465e6 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:54.520114 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-9b754d69-dcb3-4174-807c-96e5c56465e6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 14:32:54.520114 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:54.520114 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:54.520114 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:54.520114 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:54.520114 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:54.520114 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:54.520114 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:54.520114 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:54.520114 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:54.520114 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:54.520114 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:54.520114 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:32:54.520114 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:32:54.520114 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:32:54.520114 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:32:54.521539 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:54.521539 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:54.521539 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:32:54.521539 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:32:54.521539 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:32:54.521539 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:32:54.521539 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:54.521539 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:54.521539 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:54.521539 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:54.521539 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:54.521539 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:54.521539 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:54.521539 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:54.521539 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:54.521539 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:54.521539 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:54.521539 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:54.522721 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:54.522721 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:54.522721 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:32:54.522721 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:32:54.522721 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:32:54.522721 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:32:54.522721 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:32:54.522721 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:32:54.522721 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 14:32:54.522721 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:32:54.522721 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 622/1239] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:32:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:32:54.529620 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e8dd0c-df7c-430c-b8ff-3e0ffaa8c6aa None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:54.647774 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e8dd0c-df7c-430c-b8ff-3e0ffaa8c6aa None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:54.687834 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b16770-d6a7-4e1c-9321-4a1ebb1e6f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:54.688402 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b16770-d6a7-4e1c-9321-4a1ebb1e6f34 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:54.688732 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b16770-d6a7-4e1c-9321-4a1ebb1e6f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:54.689038 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b16770-d6a7-4e1c-9321-4a1ebb1e6f34 None admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:54.689844 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b16770-d6a7-4e1c-9321-4a1ebb1e6f34 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:54.690194 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b16770-d6a7-4e1c-9321-4a1ebb1e6f34 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:54.691778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b16770-d6a7-4e1c-9321-4a1ebb1e6f34 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:54.711244 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 618/1240] 10.222.0.30 () {56 vars in 1143 bytes} [Tue Aug 30 14:32:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:32:54.717389 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b754d69-dcb3-4174-807c-96e5c56465e6 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:54.837385 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b754d69-dcb3-4174-807c-96e5c56465e6 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:54.883077 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e9696-e844-4646-93c3-e5087691fda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:54.883754 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e9696-e844-4646-93c3-e5087691fda2 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:54.884081 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e9696-e844-4646-93c3-e5087691fda2 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:54.884479 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e9696-e844-4646-93c3-e5087691fda2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:54.956926 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:54.965035 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4e9696-e844-4646-93c3-e5087691fda2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:54.966428 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4e9696-e844-4646-93c3-e5087691fda2 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:55.071456 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 623/1241] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:32:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:32:55.077272 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b16770-d6a7-4e1c-9321-4a1ebb1e6f34 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:55.108612 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:55.186893 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b16770-d6a7-4e1c-9321-4a1ebb1e6f34 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:55.227109 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa0659c-1d9b-458c-b41a-2bf1d59f443b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:55.227707 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa0659c-1d9b-458c-b41a-2bf1d59f443b None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:55.227990 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa0659c-1d9b-458c-b41a-2bf1d59f443b None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:55.228262 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa0659c-1d9b-458c-b41a-2bf1d59f443b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:55.241256 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa0659c-1d9b-458c-b41a-2bf1d59f443b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:55.242711 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa0659c-1d9b-458c-b41a-2bf1d59f443b None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:55.264541 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-caa0659c-1d9b-458c-b41a-2bf1d59f443b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:32:55.264541 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:55.264541 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:55.264541 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:55.264541 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:55.264541 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:55.264541 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:55.264541 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:55.264541 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:55.264541 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:55.264541 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:55.264541 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:55.264541 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:32:55.264541 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:32:55.264541 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:32:55.264541 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:32:55.266559 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:55.266559 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:55.266559 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:55.266559 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:55.266559 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:55.266559 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:55.266559 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:55.266559 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:55.266559 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:55.266559 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:55.266559 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:55.266559 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:55.266559 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:55.266559 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:55.266559 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:55.266559 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:55.266559 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:32:55.267608 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:32:55.267608 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:32:55.267608 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:32:55.267608 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 619/1242] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:32:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:32:55.276883 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e9696-e844-4646-93c3-e5087691fda2 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:55.393420 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:55.400786 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e9696-e844-4646-93c3-e5087691fda2 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:55.444937 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2b720e-315b-41fe-8605-fca313f2860f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:55.445558 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2b720e-315b-41fe-8605-fca313f2860f None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:55.445838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2b720e-315b-41fe-8605-fca313f2860f None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:55.446170 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2b720e-315b-41fe-8605-fca313f2860f None admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:55.446951 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2b720e-315b-41fe-8605-fca313f2860f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:55.447413 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2b720e-315b-41fe-8605-fca313f2860f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:55.448977 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2b720e-315b-41fe-8605-fca313f2860f None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:55.462972 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 624/1243] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:32:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:55.469985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa0659c-1d9b-458c-b41a-2bf1d59f443b None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:55.558434 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:55.583818 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa0659c-1d9b-458c-b41a-2bf1d59f443b None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:55.619382 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a34e35-76db-492b-9c52-666737f8cc5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:55.620040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a34e35-76db-492b-9c52-666737f8cc5a None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:55.620342 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a34e35-76db-492b-9c52-666737f8cc5a None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:55.620550 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a34e35-76db-492b-9c52-666737f8cc5a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:55.627282 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a34e35-76db-492b-9c52-666737f8cc5a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:55.629085 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a34e35-76db-492b-9c52-666737f8cc5a None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:55.641675 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-75a34e35-76db-492b-9c52-666737f8cc5a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:32:55.641675 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:55.641675 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:55.641675 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:55.641675 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:55.641675 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:55.641675 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:55.641675 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:55.641675 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:55.641675 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:55.641675 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:55.641675 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:55.641675 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:32:55.641675 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:32:55.641675 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:32:55.642893 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:32:55.642893 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:55.642893 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:55.642893 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:55.642893 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:55.642893 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:55.642893 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:55.642893 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:55.642893 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:55.642893 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:55.642893 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:55.642893 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:55.642893 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:55.642893 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:55.642893 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:55.642893 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:55.642893 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:55.643837 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:32:55.643837 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:32:55.643837 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:32:55.643837 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:32:55.643837 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:32:55.643837 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:32:55.643837 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:32:55.643837 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:32:55.643837 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 620/1244] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:32:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:32:55.649964 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2b720e-315b-41fe-8605-fca313f2860f None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:55.782269 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2b720e-315b-41fe-8605-fca313f2860f None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:55.824507 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db38067-0675-4ed1-9a32-0764dfd6b12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:55.825579 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db38067-0675-4ed1-9a32-0764dfd6b12c None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:55.825850 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db38067-0675-4ed1-9a32-0764dfd6b12c None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:55.826121 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db38067-0675-4ed1-9a32-0764dfd6b12c None admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:55.826874 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db38067-0675-4ed1-9a32-0764dfd6b12c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:55.827177 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db38067-0675-4ed1-9a32-0764dfd6b12c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:55.828737 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db38067-0675-4ed1-9a32-0764dfd6b12c None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:55.844577 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 625/1245] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 30 14:32:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:55.852108 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a34e35-76db-492b-9c52-666737f8cc5a None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:55.971193 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a34e35-76db-492b-9c52-666737f8cc5a None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:56.015353 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e6c45-cf4a-412d-8cfb-f0ee34ec15ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:56.016017 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e6c45-cf4a-412d-8cfb-f0ee34ec15ca None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:56.016302 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e6c45-cf4a-412d-8cfb-f0ee34ec15ca None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:56.016568 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e6c45-cf4a-412d-8cfb-f0ee34ec15ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:56.017303 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306e6c45-cf4a-412d-8cfb-f0ee34ec15ca None admin] RBAC: Adding query filter params (role.id=1316db16eac54629a283545f1601a236, scope.project.id=01cf71ca5abf4576b89d10c85af6d2de, user.id=dcf302a170934c25b39699088dac326a) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:56.017613 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306e6c45-cf4a-412d-8cfb-f0ee34ec15ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:56.019035 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306e6c45-cf4a-412d-8cfb-f0ee34ec15ca None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:56.040459 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 621/1246] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:32:55 2022] GET /identity/v3/role_assignments?user.id=dcf302a170934c25b39699088dac326a&scope.project.id=01cf71ca5abf4576b89d10c85af6d2de&role.id=1316db16eac54629a283545f1601a236 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:32:58.099290 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917bf7ea-40ba-4091-8c9c-16ceb8a23cda None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:58.100178 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917bf7ea-40ba-4091-8c9c-16ceb8a23cda None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:58.100178 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917bf7ea-40ba-4091-8c9c-16ceb8a23cda None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:58.101945 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 626/1247] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:32:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:32:58.109342 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6056de33-aaa7-42e2-b8a8-faa39d4fe16c None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:58.109935 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6056de33-aaa7-42e2-b8a8-faa39d4fe16c None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:58.109935 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6056de33-aaa7-42e2-b8a8-faa39d4fe16c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:58.123010 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:58.140220 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-6056de33-aaa7-42e2-b8a8-faa39d4fe16c None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:32:58.141760 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-6056de33-aaa7-42e2-b8a8-faa39d4fe16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:32:58.225313 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 622/1248] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:32:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:32:58.543255 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917bf7ea-40ba-4091-8c9c-16ceb8a23cda None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:58.566875 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:58.665563 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917bf7ea-40ba-4091-8c9c-16ceb8a23cda None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:58.701833 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db098421-3a16-4dc8-bcfd-2eee255847fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:58.702654 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db098421-3a16-4dc8-bcfd-2eee255847fe None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:58.703058 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db098421-3a16-4dc8-bcfd-2eee255847fe None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:58.703558 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db098421-3a16-4dc8-bcfd-2eee255847fe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:58.716600 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db098421-3a16-4dc8-bcfd-2eee255847fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:32:58.721464 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-db098421-3a16-4dc8-bcfd-2eee255847fe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:32:58.721464 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:58.721464 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:58.721464 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:58.721464 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:58.721464 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:58.721464 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:58.721464 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:58.721464 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:58.721464 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:58.721464 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:58.721464 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:58.721464 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:32:58.721464 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:32:58.721464 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:32:58.721464 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:32:58.723190 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:32:58.723190 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:32:58.723190 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:32:58.723190 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:32:58.723190 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:58.723190 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:58.723190 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:58.723190 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:58.723190 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:58.723190 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:58.723190 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:58.723190 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:58.723190 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:58.723190 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:58.723190 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:58.723190 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:58.724103 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:58.724103 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:58.724103 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:58.724103 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:58.724103 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:32:58.724103 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:32:58.724103 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:32:58.724103 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:32:58.724103 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:32:58.724103 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:32:58.724103 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:32:58.724103 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:32:58.724103 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 627/1249] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 30 14:32:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:32:58.729214 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6056de33-aaa7-42e2-b8a8-faa39d4fe16c None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:58.834844 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6056de33-aaa7-42e2-b8a8-faa39d4fe16c None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:58.867640 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:58.872938 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8b47c7-facd-452b-b0fe-e3d06edca760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:58.873571 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8b47c7-facd-452b-b0fe-e3d06edca760 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:58.873842 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8b47c7-facd-452b-b0fe-e3d06edca760 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:58.874110 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8b47c7-facd-452b-b0fe-e3d06edca760 None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:58.874760 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8b47c7-facd-452b-b0fe-e3d06edca760 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:58.875055 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8b47c7-facd-452b-b0fe-e3d06edca760 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:58.876432 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8b47c7-facd-452b-b0fe-e3d06edca760 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:58.884474 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 623/1250] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:32:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:32:58.890209 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db098421-3a16-4dc8-bcfd-2eee255847fe None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:58.977134 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:59.004966 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db098421-3a16-4dc8-bcfd-2eee255847fe None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:59.039929 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b6d69-eee9-448f-b6ca-6e629dab2adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:59.040561 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b6d69-eee9-448f-b6ca-6e629dab2adb None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:59.040843 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b6d69-eee9-448f-b6ca-6e629dab2adb None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:59.041114 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b6d69-eee9-448f-b6ca-6e629dab2adb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:59.051017 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708b6d69-eee9-448f-b6ca-6e629dab2adb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:59.052507 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708b6d69-eee9-448f-b6ca-6e629dab2adb None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:59.067730 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-708b6d69-eee9-448f-b6ca-6e629dab2adb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:32:59.067730 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:59.067730 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:59.067730 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:59.067730 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:59.067730 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:59.067730 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:59.067730 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:59.067730 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:59.067730 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:59.067730 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:59.067730 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:59.067730 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:32:59.067730 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:32:59.067730 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:32:59.067730 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:32:59.069314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:59.069314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:59.069314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:59.069314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:59.069314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:59.069314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:59.069314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:59.069314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:59.069314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:59.069314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:59.069314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:59.069314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:59.069314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:59.069314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:59.069314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:59.069314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:59.069314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:32:59.070330 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:32:59.070330 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:32:59.070330 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:32:59.070330 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 628/1251] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:32:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:32:59.074902 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8b47c7-facd-452b-b0fe-e3d06edca760 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:59.186120 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8b47c7-facd-452b-b0fe-e3d06edca760 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:59.225137 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c58c3c-4ae5-4d63-bf27-17d8a787450f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:59.225822 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c58c3c-4ae5-4d63-bf27-17d8a787450f None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:59.226096 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c58c3c-4ae5-4d63-bf27-17d8a787450f None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:59.226366 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c58c3c-4ae5-4d63-bf27-17d8a787450f None admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:59.227037 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c58c3c-4ae5-4d63-bf27-17d8a787450f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:59.227378 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c58c3c-4ae5-4d63-bf27-17d8a787450f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:59.228924 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c58c3c-4ae5-4d63-bf27-17d8a787450f None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:59.247240 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 624/1252] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:32:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:32:59.249175 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b6d69-eee9-448f-b6ca-6e629dab2adb None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:59.366790 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b6d69-eee9-448f-b6ca-6e629dab2adb None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:59.396477 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:59.405370 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ff3b4-4a61-409d-9aad-02ef58d5f4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:59.405960 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ff3b4-4a61-409d-9aad-02ef58d5f4cb None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:59.406244 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ff3b4-4a61-409d-9aad-02ef58d5f4cb None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:59.406510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ff3b4-4a61-409d-9aad-02ef58d5f4cb None admin] PATH_INFO: `/v3/users/placement` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:59.414849 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ff3b4-4a61-409d-9aad-02ef58d5f4cb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:59.416478 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ff3b4-4a61-409d-9aad-02ef58d5f4cb None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:59.428983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-7d0ff3b4-4a61-409d-9aad-02ef58d5f4cb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 14:32:59.428983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:32:59.428983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:32:59.428983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:32:59.428983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:32:59.428983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:32:59.428983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:32:59.428983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:32:59.428983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:32:59.428983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:32:59.428983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:32:59.428983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:32:59.428983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:32:59.428983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:32:59.428983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:32:59.428983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:32:59.430090 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:32:59.430090 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:32:59.430090 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:32:59.430090 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:32:59.430090 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:32:59.430090 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:32:59.430090 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:32:59.430090 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:32:59.430090 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:32:59.430090 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:32:59.430090 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:32:59.430090 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:32:59.430090 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:32:59.430090 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:32:59.430090 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:32:59.430090 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:32:59.430090 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:32:59.430090 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:32:59.431254 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:32:59.431254 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:32:59.431254 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:32:59.431254 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:32:59.431254 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:32:59.431254 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:32:59.431254 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:32:59.431254 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:32:59.431254 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 14:32:59.431254 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:32:59.432634 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 629/1253] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:32:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 14:32:59.439554 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c58c3c-4ae5-4d63-bf27-17d8a787450f None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:59.505221 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:59.554193 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c58c3c-4ae5-4d63-bf27-17d8a787450f None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:59.593843 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8868d8be-d861-4c9b-a4e1-22b7c3b213a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:59.594609 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868d8be-d861-4c9b-a4e1-22b7c3b213a8 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:59.594889 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868d8be-d861-4c9b-a4e1-22b7c3b213a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:59.595308 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868d8be-d861-4c9b-a4e1-22b7c3b213a8 None admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:59.596275 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8868d8be-d861-4c9b-a4e1-22b7c3b213a8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:32:59.596637 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8868d8be-d861-4c9b-a4e1-22b7c3b213a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:59.598533 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8868d8be-d861-4c9b-a4e1-22b7c3b213a8 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:59.623876 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 625/1254] 10.222.0.30 () {56 vars in 1143 bytes} [Tue Aug 30 14:32:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:32:59.625239 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ff3b4-4a61-409d-9aad-02ef58d5f4cb None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:32:59.734534 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ff3b4-4a61-409d-9aad-02ef58d5f4cb None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:32:59.767688 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebcbc35-bf9d-4f80-8c1c-81994ce27753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:32:59.768290 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebcbc35-bf9d-4f80-8c1c-81994ce27753 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:32:59.768571 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebcbc35-bf9d-4f80-8c1c-81994ce27753 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:32:59.768844 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebcbc35-bf9d-4f80-8c1c-81994ce27753 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:32:59.828883 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:32:59.842126 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebcbc35-bf9d-4f80-8c1c-81994ce27753 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:32:59.843655 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebcbc35-bf9d-4f80-8c1c-81994ce27753 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:32:59.958235 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 630/1255] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:32:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 14:32:59.964267 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8868d8be-d861-4c9b-a4e1-22b7c3b213a8 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:00.073734 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8868d8be-d861-4c9b-a4e1-22b7c3b213a8 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:00.088150 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:00.111706 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb551e-1fec-4a0b-a3bc-0dc218c90b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:00.112425 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb551e-1fec-4a0b-a3bc-0dc218c90b49 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:00.112714 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb551e-1fec-4a0b-a3bc-0dc218c90b49 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:00.113148 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb551e-1fec-4a0b-a3bc-0dc218c90b49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:00.123942 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcb551e-1fec-4a0b-a3bc-0dc218c90b49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:00.125433 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcb551e-1fec-4a0b-a3bc-0dc218c90b49 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:00.140778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-2fcb551e-1fec-4a0b-a3bc-0dc218c90b49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:33:00.140778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:33:00.140778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:33:00.140778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:33:00.140778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:33:00.140778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:33:00.140778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:33:00.140778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:33:00.140778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:33:00.140778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:33:00.140778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:33:00.140778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:33:00.140778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:33:00.140778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:33:00.140778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:33:00.140778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:33:00.141880 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:33:00.141880 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:33:00.141880 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:33:00.141880 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:33:00.141880 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:33:00.141880 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:33:00.141880 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:33:00.141880 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:33:00.141880 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:33:00.141880 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:33:00.141880 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:33:00.141880 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:33:00.141880 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:33:00.141880 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:33:00.141880 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:33:00.141880 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:33:00.141880 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:33:00.142821 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:33:00.142821 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:33:00.142821 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:33:00.142821 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 626/1256] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:32:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:33:00.149773 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebcbc35-bf9d-4f80-8c1c-81994ce27753 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:00.257780 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:00.272961 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebcbc35-bf9d-4f80-8c1c-81994ce27753 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:00.317708 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799e7a7b-1d3d-4aa6-9e69-c74c9b1734c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:00.318855 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e7a7b-1d3d-4aa6-9e69-c74c9b1734c5 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:00.319198 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e7a7b-1d3d-4aa6-9e69-c74c9b1734c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:00.319650 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e7a7b-1d3d-4aa6-9e69-c74c9b1734c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:00.320349 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799e7a7b-1d3d-4aa6-9e69-c74c9b1734c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:33:00.320693 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799e7a7b-1d3d-4aa6-9e69-c74c9b1734c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:00.322932 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799e7a7b-1d3d-4aa6-9e69-c74c9b1734c5 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:00.342708 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 631/1257] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:33:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:33:00.349982 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb551e-1fec-4a0b-a3bc-0dc218c90b49 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:00.468713 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb551e-1fec-4a0b-a3bc-0dc218c90b49 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:00.505772 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf55e6-1034-4d25-887f-245e60bc08cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:00.506455 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf55e6-1034-4d25-887f-245e60bc08cd None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:00.506734 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf55e6-1034-4d25-887f-245e60bc08cd None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:00.507006 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf55e6-1034-4d25-887f-245e60bc08cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:00.516798 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbf55e6-1034-4d25-887f-245e60bc08cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:00.518464 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbf55e6-1034-4d25-887f-245e60bc08cd None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:00.532058 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-abbf55e6-1034-4d25-887f-245e60bc08cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:33:00.532058 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:33:00.532058 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:33:00.532058 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:33:00.532058 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:33:00.532058 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:33:00.532058 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:33:00.532058 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:33:00.532058 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:33:00.532058 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:33:00.532058 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:33:00.532058 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:33:00.532058 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:33:00.532058 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:33:00.532058 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:33:00.533204 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:33:00.533204 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:33:00.533204 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:33:00.533204 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:33:00.533204 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:33:00.533204 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:33:00.533204 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:33:00.533204 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:33:00.533204 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:33:00.533204 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:33:00.533204 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:33:00.533204 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:33:00.533204 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:33:00.533204 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:33:00.533204 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:33:00.533204 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:33:00.533204 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:33:00.534173 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:33:00.534173 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:33:00.534173 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:33:00.534173 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:33:00.534173 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:33:00.534173 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:33:00.534173 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:33:00.534173 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:33:00.534173 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 627/1258] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:33:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:33:00.539492 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799e7a7b-1d3d-4aa6-9e69-c74c9b1734c5 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:00.647666 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799e7a7b-1d3d-4aa6-9e69-c74c9b1734c5 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:00.683637 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04786bdb-6d4c-4b5a-8339-7f6c5520c858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:00.684533 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04786bdb-6d4c-4b5a-8339-7f6c5520c858 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:00.684818 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04786bdb-6d4c-4b5a-8339-7f6c5520c858 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:00.685130 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04786bdb-6d4c-4b5a-8339-7f6c5520c858 None admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:00.685844 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04786bdb-6d4c-4b5a-8339-7f6c5520c858 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:33:00.686149 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04786bdb-6d4c-4b5a-8339-7f6c5520c858 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:00.687639 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04786bdb-6d4c-4b5a-8339-7f6c5520c858 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:00.701055 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 632/1259] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 30 14:33:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:33:00.707739 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf55e6-1034-4d25-887f-245e60bc08cd None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:00.735222 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:00.813781 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf55e6-1034-4d25-887f-245e60bc08cd None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:00.849785 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae641704-699a-4a13-9df7-90a829020b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:00.850459 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae641704-699a-4a13-9df7-90a829020b63 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:00.850736 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae641704-699a-4a13-9df7-90a829020b63 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:00.851000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae641704-699a-4a13-9df7-90a829020b63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:00.851760 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae641704-699a-4a13-9df7-90a829020b63 None admin] RBAC: Adding query filter params (role.id=3cda0615d87d4af0860a2a5e9b4e2c05, scope.project.id=01cf71ca5abf4576b89d10c85af6d2de, user.id=dcf302a170934c25b39699088dac326a) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:33:00.852061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae641704-699a-4a13-9df7-90a829020b63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:00.853508 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae641704-699a-4a13-9df7-90a829020b63 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:00.859553 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 628/1260] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:33:00 2022] GET /identity/v3/role_assignments?user.id=dcf302a170934c25b39699088dac326a&scope.project.id=01cf71ca5abf4576b89d10c85af6d2de&role.id=3cda0615d87d4af0860a2a5e9b4e2c05 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:33:02.723794 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f7262-79d2-4a25-a622-c8ddd95be994 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:02.724325 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f7262-79d2-4a25-a622-c8ddd95be994 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:02.724325 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f7262-79d2-4a25-a622-c8ddd95be994 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:02.725671 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 633/1261] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:33:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:33:02.730767 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0941eaf-077d-4d21-8237-077faea075ce None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:02.731183 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0941eaf-077d-4d21-8237-077faea075ce None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:02.731333 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0941eaf-077d-4d21-8237-077faea075ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:02.749229 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-c0941eaf-077d-4d21-8237-077faea075ce None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:33:02.750428 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-c0941eaf-077d-4d21-8237-077faea075ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:33:02.815922 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 629/1262] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:33:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:33:03.086886 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1f7262-79d2-4a25-a622-c8ddd95be994 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:03.097876 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:03.190341 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1f7262-79d2-4a25-a622-c8ddd95be994 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:03.224215 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9717b152-4585-424d-ae6a-bc3084232d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:03.224832 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717b152-4585-424d-ae6a-bc3084232d7d None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:03.225106 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717b152-4585-424d-ae6a-bc3084232d7d None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:03.225372 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717b152-4585-424d-ae6a-bc3084232d7d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:03.235056 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9717b152-4585-424d-ae6a-bc3084232d7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:33:03.240670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-9717b152-4585-424d-ae6a-bc3084232d7d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:33:03.240670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:33:03.240670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:33:03.240670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:33:03.240670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:33:03.240670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:33:03.240670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:33:03.240670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:33:03.240670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:33:03.240670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:33:03.240670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:33:03.240670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:33:03.240670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:33:03.240670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:33:03.240670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:33:03.240670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:33:03.241617 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:33:03.241617 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:33:03.241617 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:33:03.241617 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:33:03.241617 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:33:03.241617 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:33:03.241617 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:33:03.241617 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:33:03.241617 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:33:03.241617 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:33:03.241617 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:33:03.241617 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:33:03.241617 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:33:03.241617 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:33:03.241617 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:33:03.241617 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:33:03.242645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:33:03.242645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:33:03.242645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:33:03.242645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:33:03.242645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:33:03.242645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:33:03.242645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:33:03.242645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:33:03.242645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:33:03.242645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:33:03.242645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:33:03.242645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:33:03.242645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 634/1263] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 30 14:33:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:33:03.247662 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0941eaf-077d-4d21-8237-077faea075ce None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:03.266129 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:03.349985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0941eaf-077d-4d21-8237-077faea075ce None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:03.391422 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b774c29-26a4-47ab-89c1-efc58e621441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:03.392025 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b774c29-26a4-47ab-89c1-efc58e621441 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:03.392298 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b774c29-26a4-47ab-89c1-efc58e621441 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:03.392563 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b774c29-26a4-47ab-89c1-efc58e621441 None admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:03.393209 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b774c29-26a4-47ab-89c1-efc58e621441 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:33:03.393504 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b774c29-26a4-47ab-89c1-efc58e621441 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:03.394850 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b774c29-26a4-47ab-89c1-efc58e621441 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:03.402647 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 630/1264] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:33:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:33:03.408380 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9717b152-4585-424d-ae6a-bc3084232d7d None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:03.475022 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:03.507984 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9717b152-4585-424d-ae6a-bc3084232d7d None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:03.540972 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497ea8ed-cac0-4708-b7e2-1b4c1171d728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:03.541596 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ea8ed-cac0-4708-b7e2-1b4c1171d728 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:03.541873 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ea8ed-cac0-4708-b7e2-1b4c1171d728 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:03.542143 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ea8ed-cac0-4708-b7e2-1b4c1171d728 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:03.552433 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497ea8ed-cac0-4708-b7e2-1b4c1171d728 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:03.553864 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497ea8ed-cac0-4708-b7e2-1b4c1171d728 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:03.568231 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-497ea8ed-cac0-4708-b7e2-1b4c1171d728 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:33:03.568231 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:33:03.568231 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:33:03.568231 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:33:03.568231 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:33:03.568231 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:33:03.568231 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:33:03.568231 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:33:03.568231 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:33:03.568231 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:33:03.568231 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:33:03.568231 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:33:03.568231 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:33:03.568231 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:33:03.568231 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:33:03.568231 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:33:03.569363 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:33:03.569363 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:33:03.569363 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:33:03.569363 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:33:03.569363 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:33:03.569363 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:33:03.569363 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:33:03.569363 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:33:03.569363 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:33:03.569363 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:33:03.569363 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:33:03.569363 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:33:03.569363 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:33:03.569363 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:33:03.569363 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:33:03.569363 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:33:03.569363 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:33:03.570294 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:33:03.570294 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:33:03.570294 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:33:03.570294 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 635/1265] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:33:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:33:03.575276 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b774c29-26a4-47ab-89c1-efc58e621441 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:03.653221 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:03.672153 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b774c29-26a4-47ab-89c1-efc58e621441 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:03.704525 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d43d14-b237-413f-bb60-d97ae5701955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:03.705116 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d43d14-b237-413f-bb60-d97ae5701955 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:03.705391 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d43d14-b237-413f-bb60-d97ae5701955 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:03.705657 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d43d14-b237-413f-bb60-d97ae5701955 None admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:03.706281 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d43d14-b237-413f-bb60-d97ae5701955 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:33:03.706576 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d43d14-b237-413f-bb60-d97ae5701955 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:03.707974 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d43d14-b237-413f-bb60-d97ae5701955 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:03.719084 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 631/1266] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:33:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:33:03.725211 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497ea8ed-cac0-4708-b7e2-1b4c1171d728 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:03.823476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497ea8ed-cac0-4708-b7e2-1b4c1171d728 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:03.843130 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:03.858073 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363ca5a6-392c-44cb-8c96-a937d38187fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:03.858687 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363ca5a6-392c-44cb-8c96-a937d38187fc None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:03.858970 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363ca5a6-392c-44cb-8c96-a937d38187fc None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:03.859274 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363ca5a6-392c-44cb-8c96-a937d38187fc None admin] PATH_INFO: `/v3/users/placement` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:03.866475 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363ca5a6-392c-44cb-8c96-a937d38187fc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:03.867957 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363ca5a6-392c-44cb-8c96-a937d38187fc None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:03.878806 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-363ca5a6-392c-44cb-8c96-a937d38187fc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 14:33:03.878806 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:33:03.878806 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:33:03.878806 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:33:03.878806 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:33:03.878806 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:33:03.878806 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:33:03.878806 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:33:03.878806 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:33:03.878806 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:33:03.878806 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:33:03.878806 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:33:03.878806 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:33:03.878806 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:33:03.878806 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:33:03.878806 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:33:03.879960 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:33:03.879960 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:33:03.879960 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:33:03.879960 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:33:03.879960 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:33:03.879960 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:33:03.879960 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:33:03.879960 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:33:03.879960 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:33:03.879960 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:33:03.879960 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:33:03.879960 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:33:03.879960 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:33:03.879960 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:33:03.879960 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:33:03.879960 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:33:03.879960 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:33:03.879960 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:33:03.881057 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:33:03.881057 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:33:03.881057 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:33:03.881057 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:33:03.881057 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:33:03.881057 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:33:03.881057 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:33:03.881057 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:33:03.881057 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 14:33:03.881057 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:33:03.881057 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 636/1267] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:33:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:33:03.885625 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d43d14-b237-413f-bb60-d97ae5701955 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:03.982153 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d43d14-b237-413f-bb60-d97ae5701955 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:04.014407 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023de121-0cad-41cb-9d8d-5e0b5d0f6192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:04.015002 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023de121-0cad-41cb-9d8d-5e0b5d0f6192 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:04.015308 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023de121-0cad-41cb-9d8d-5e0b5d0f6192 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:04.015583 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023de121-0cad-41cb-9d8d-5e0b5d0f6192 None admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:04.016301 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023de121-0cad-41cb-9d8d-5e0b5d0f6192 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:33:04.016602 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023de121-0cad-41cb-9d8d-5e0b5d0f6192 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:04.018025 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023de121-0cad-41cb-9d8d-5e0b5d0f6192 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:04.026901 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:04.035972 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 632/1268] 10.222.0.30 () {56 vars in 1143 bytes} [Tue Aug 30 14:33:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:33:04.041450 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363ca5a6-392c-44cb-8c96-a937d38187fc None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:04.140297 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363ca5a6-392c-44cb-8c96-a937d38187fc None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:04.172859 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16dd1a5-82dd-4e55-8122-ada1b7632aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:04.173468 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16dd1a5-82dd-4e55-8122-ada1b7632aa9 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:04.173749 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16dd1a5-82dd-4e55-8122-ada1b7632aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:04.174024 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16dd1a5-82dd-4e55-8122-ada1b7632aa9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:04.184995 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16dd1a5-82dd-4e55-8122-ada1b7632aa9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:04.186427 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16dd1a5-82dd-4e55-8122-ada1b7632aa9 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:04.200756 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-a16dd1a5-82dd-4e55-8122-ada1b7632aa9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:33:04.200756 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:33:04.200756 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:33:04.200756 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:33:04.200756 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:33:04.200756 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:33:04.200756 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:33:04.200756 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:33:04.200756 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:33:04.200756 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:33:04.200756 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:33:04.200756 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:33:04.200756 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:33:04.200756 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:33:04.200756 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:33:04.200756 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:33:04.201716 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:33:04.201716 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:33:04.201716 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:33:04.201716 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:33:04.201716 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:33:04.201716 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:33:04.201716 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:33:04.201716 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:33:04.201716 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:33:04.201716 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:33:04.201716 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:33:04.201716 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:33:04.201716 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:33:04.201716 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:33:04.201716 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:33:04.201716 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:33:04.201716 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:33:04.202708 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:33:04.202708 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:33:04.202708 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:33:04.202708 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 637/1269] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:33:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:33:04.207688 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023de121-0cad-41cb-9d8d-5e0b5d0f6192 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:04.305850 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023de121-0cad-41cb-9d8d-5e0b5d0f6192 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:04.343588 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe3dfd3-9c3c-4eb7-90db-7872a90cd91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:04.344215 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3dfd3-9c3c-4eb7-90db-7872a90cd91b None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:04.344533 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3dfd3-9c3c-4eb7-90db-7872a90cd91b None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:04.344847 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3dfd3-9c3c-4eb7-90db-7872a90cd91b None admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:04.345606 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe3dfd3-9c3c-4eb7-90db-7872a90cd91b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:33:04.345942 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe3dfd3-9c3c-4eb7-90db-7872a90cd91b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:04.347982 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe3dfd3-9c3c-4eb7-90db-7872a90cd91b None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:04.359463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 633/1270] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:33:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:33:04.365837 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16dd1a5-82dd-4e55-8122-ada1b7632aa9 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:04.397182 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:04.464824 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16dd1a5-82dd-4e55-8122-ada1b7632aa9 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:04.497630 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcb8dfe-be38-43b4-8d43-e13cbbd2ee3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:04.498240 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb8dfe-be38-43b4-8d43-e13cbbd2ee3f None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:04.498521 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb8dfe-be38-43b4-8d43-e13cbbd2ee3f None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:04.498795 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb8dfe-be38-43b4-8d43-e13cbbd2ee3f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:04.505541 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcb8dfe-be38-43b4-8d43-e13cbbd2ee3f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:04.506962 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcb8dfe-be38-43b4-8d43-e13cbbd2ee3f None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:04.517815 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-2bcb8dfe-be38-43b4-8d43-e13cbbd2ee3f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:33:04.517815 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:33:04.517815 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:33:04.517815 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:33:04.517815 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:33:04.517815 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:33:04.517815 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:33:04.517815 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:33:04.517815 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:33:04.517815 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:33:04.517815 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:33:04.517815 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:33:04.517815 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:33:04.517815 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:33:04.517815 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:33:04.518854 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:33:04.518854 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:33:04.518854 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:33:04.518854 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:33:04.518854 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:33:04.518854 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:33:04.518854 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:33:04.518854 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:33:04.518854 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:33:04.518854 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:33:04.518854 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:33:04.518854 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:33:04.518854 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:33:04.518854 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:33:04.518854 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:33:04.518854 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:33:04.518854 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:33:04.520004 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:33:04.520004 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:33:04.520004 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:33:04.520004 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:33:04.520004 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:33:04.520004 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:33:04.520004 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:33:04.520004 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:33:04.520004 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 638/1271] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:33:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:33:04.524876 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe3dfd3-9c3c-4eb7-90db-7872a90cd91b None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:04.582746 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:04.623491 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe3dfd3-9c3c-4eb7-90db-7872a90cd91b None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:04.656194 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438cbaf9-10a6-4568-bd0c-7251d9cdfaf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:04.656787 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438cbaf9-10a6-4568-bd0c-7251d9cdfaf3 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:04.657065 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438cbaf9-10a6-4568-bd0c-7251d9cdfaf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:04.657336 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438cbaf9-10a6-4568-bd0c-7251d9cdfaf3 None admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:04.658016 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438cbaf9-10a6-4568-bd0c-7251d9cdfaf3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:33:04.658318 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438cbaf9-10a6-4568-bd0c-7251d9cdfaf3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:04.660394 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438cbaf9-10a6-4568-bd0c-7251d9cdfaf3 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:04.672033 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 634/1272] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 30 14:33:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:33:04.678173 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcb8dfe-be38-43b4-8d43-e13cbbd2ee3f None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:04.766843 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:04.776245 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcb8dfe-be38-43b4-8d43-e13cbbd2ee3f None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:04.809224 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f312ae-83cd-4ba5-9249-729893ace7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:04.809860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f312ae-83cd-4ba5-9249-729893ace7d8 None admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:04.810140 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f312ae-83cd-4ba5-9249-729893ace7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:04.810411 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f312ae-83cd-4ba5-9249-729893ace7d8 None admin] PATH_INFO: `/v3/projects/01cf71ca5abf4576b89d10c85af6d2de/users/dcf302a170934c25b39699088dac326a/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:04.812298 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f312ae-83cd-4ba5-9249-729893ace7d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=01cf71ca5abf4576b89d10c85af6d2de, user_id=dcf302a170934c25b39699088dac326a, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:04.813707 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f312ae-83cd-4ba5-9249-729893ace7d8 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:04.825885 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 639/1273] 10.222.0.30 () {58 vars in 1439 bytes} [Tue Aug 30 14:33:04 2022] PUT /identity/v3/projects/01cf71ca5abf4576b89d10c85af6d2de/users/dcf302a170934c25b39699088dac326a/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:33:06.708369 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1591549-9a0d-4046-9bfa-b32b0ab48e3d None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:06.708960 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1591549-9a0d-4046-9bfa-b32b0ab48e3d None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:06.708960 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1591549-9a0d-4046-9bfa-b32b0ab48e3d None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:06.710254 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 635/1274] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:33:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:33:06.715378 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b4c0f-fdad-4adb-95b1-5a7cfa079030 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:06.715776 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b4c0f-fdad-4adb-95b1-5a7cfa079030 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:06.715920 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b4c0f-fdad-4adb-95b1-5a7cfa079030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:06.733692 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-6c1b4c0f-fdad-4adb-95b1-5a7cfa079030 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:33:06.734866 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-6c1b4c0f-fdad-4adb-95b1-5a7cfa079030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:33:06.799055 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 640/1275] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:33:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:33:07.065259 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1591549-9a0d-4046-9bfa-b32b0ab48e3d None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:07.163625 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1591549-9a0d-4046-9bfa-b32b0ab48e3d None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:07.180655 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:07.198314 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742cd4e4-fccf-4181-b473-eddec1b1a01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:07.198937 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742cd4e4-fccf-4181-b473-eddec1b1a01a None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:07.199231 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742cd4e4-fccf-4181-b473-eddec1b1a01a None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:07.199519 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742cd4e4-fccf-4181-b473-eddec1b1a01a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:07.212314 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742cd4e4-fccf-4181-b473-eddec1b1a01a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 14:33:07.217101 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-742cd4e4-fccf-4181-b473-eddec1b1a01a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:33:07.217101 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:33:07.217101 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:33:07.217101 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:33:07.217101 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:33:07.217101 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:33:07.217101 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:33:07.217101 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:33:07.217101 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:33:07.217101 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:33:07.217101 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:33:07.217101 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:33:07.217101 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 14:33:07.217101 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 14:33:07.217101 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 14:33:07.217101 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 14:33:07.218358 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 14:33:07.218358 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 14:33:07.218358 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 14:33:07.218358 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 14:33:07.218358 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:33:07.218358 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:33:07.218358 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:33:07.218358 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:33:07.218358 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:33:07.218358 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:33:07.218358 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:33:07.218358 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:33:07.218358 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:33:07.218358 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:33:07.218358 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:33:07.218358 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:33:07.219456 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:33:07.219456 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:33:07.219456 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:33:07.219456 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:33:07.219456 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 14:33:07.219456 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 14:33:07.219456 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 14:33:07.219456 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 14:33:07.219456 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 14:33:07.219456 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 14:33:07.219456 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 14:33:07.219456 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:33:07.219456 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 636/1276] 10.222.0.30 () {56 vars in 1096 bytes} [Tue Aug 30 14:33:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:33:07.224792 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1b4c0f-fdad-4adb-95b1-5a7cfa079030 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:07.305437 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:07.326856 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1b4c0f-fdad-4adb-95b1-5a7cfa079030 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:07.364966 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e23e12-75ed-4d4a-9466-4773e1b00929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:07.365606 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e23e12-75ed-4d4a-9466-4773e1b00929 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:07.365924 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e23e12-75ed-4d4a-9466-4773e1b00929 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:07.366239 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e23e12-75ed-4d4a-9466-4773e1b00929 None admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:07.367037 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e23e12-75ed-4d4a-9466-4773e1b00929 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:33:07.367506 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e23e12-75ed-4d4a-9466-4773e1b00929 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:07.369207 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e23e12-75ed-4d4a-9466-4773e1b00929 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:07.378973 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 641/1277] 10.222.0.30 () {56 vars in 1099 bytes} [Tue Aug 30 14:33:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:33:07.385418 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742cd4e4-fccf-4181-b473-eddec1b1a01a None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:07.485287 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742cd4e4-fccf-4181-b473-eddec1b1a01a None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:07.519231 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc10b7a-c1bf-4ce1-a45b-985b5ebe77eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:07.519874 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc10b7a-c1bf-4ce1-a45b-985b5ebe77eb None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:07.520150 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc10b7a-c1bf-4ce1-a45b-985b5ebe77eb None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:07.520423 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc10b7a-c1bf-4ce1-a45b-985b5ebe77eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:07.530275 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc10b7a-c1bf-4ce1-a45b-985b5ebe77eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:07.531749 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc10b7a-c1bf-4ce1-a45b-985b5ebe77eb None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:07.546338 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-cbc10b7a-c1bf-4ce1-a45b-985b5ebe77eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:33:07.546338 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:33:07.546338 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:33:07.546338 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:33:07.546338 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:33:07.546338 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:33:07.546338 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:33:07.546338 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:33:07.546338 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:33:07.546338 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:33:07.546338 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:33:07.546338 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:33:07.546338 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:33:07.546338 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:33:07.546338 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:33:07.546338 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:33:07.547463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:33:07.547463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:33:07.547463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:33:07.547463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:33:07.547463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:33:07.547463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:33:07.547463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:33:07.547463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:33:07.547463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:33:07.547463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:33:07.547463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:33:07.547463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:33:07.547463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:33:07.547463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:33:07.547463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:33:07.547463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:33:07.547463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:33:07.548560 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:33:07.548560 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:33:07.548560 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:33:07.548560 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 637/1278] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:33:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:33:07.554056 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e23e12-75ed-4d4a-9466-4773e1b00929 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:07.651551 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e23e12-75ed-4d4a-9466-4773e1b00929 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:07.684582 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0373d11-4e0d-4520-9272-038084e5d29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:07.685174 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0373d11-4e0d-4520-9272-038084e5d29d None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:07.685458 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0373d11-4e0d-4520-9272-038084e5d29d None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:07.685733 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0373d11-4e0d-4520-9272-038084e5d29d None admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:07.686369 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0373d11-4e0d-4520-9272-038084e5d29d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:33:07.686671 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0373d11-4e0d-4520-9272-038084e5d29d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:07.688069 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0373d11-4e0d-4520-9272-038084e5d29d None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:07.699461 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 642/1279] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:33:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:33:07.705037 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc10b7a-c1bf-4ce1-a45b-985b5ebe77eb None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:07.736432 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:07.805177 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc10b7a-c1bf-4ce1-a45b-985b5ebe77eb None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:07.837741 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef6353b-3bae-406f-b888-600012d4d248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:07.838354 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6353b-3bae-406f-b888-600012d4d248 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:07.838631 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6353b-3bae-406f-b888-600012d4d248 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:07.838907 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6353b-3bae-406f-b888-600012d4d248 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:07.846007 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef6353b-3bae-406f-b888-600012d4d248 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:07.848009 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef6353b-3bae-406f-b888-600012d4d248 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:07.858778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-0ef6353b-3bae-406f-b888-600012d4d248 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 14:33:07.858778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:33:07.858778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:33:07.858778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:33:07.858778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:33:07.858778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:33:07.858778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:33:07.858778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:33:07.858778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:33:07.858778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:33:07.858778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:33:07.858778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:33:07.858778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 14:33:07.858778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 14:33:07.858778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 14:33:07.858778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 14:33:07.860041 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:33:07.860041 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:33:07.860041 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 14:33:07.860041 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:33:07.860041 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 14:33:07.860041 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 14:33:07.860041 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:33:07.860041 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:33:07.860041 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:33:07.860041 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:33:07.860041 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:33:07.860041 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:33:07.860041 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:33:07.860041 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:33:07.860041 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:33:07.860041 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:33:07.860041 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:33:07.860041 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:33:07.861230 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:33:07.861230 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:33:07.861230 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 14:33:07.861230 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 14:33:07.861230 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 14:33:07.861230 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 14:33:07.861230 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 14:33:07.861230 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 14:33:07.861230 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 14:33:07.861230 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:33:07.861230 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 638/1280] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:33:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:33:07.865840 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0373d11-4e0d-4520-9272-038084e5d29d None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:07.869827 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:07.963762 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0373d11-4e0d-4520-9272-038084e5d29d None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:07.996178 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0781930b-d5ba-40da-804b-75df92371ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:07.996770 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0781930b-d5ba-40da-804b-75df92371ae0 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:07.997045 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0781930b-d5ba-40da-804b-75df92371ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:07.997313 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0781930b-d5ba-40da-804b-75df92371ae0 None admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:07.998026 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0781930b-d5ba-40da-804b-75df92371ae0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:33:07.998326 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0781930b-d5ba-40da-804b-75df92371ae0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:07.999776 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0781930b-d5ba-40da-804b-75df92371ae0 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:08.016530 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 643/1281] 10.222.0.30 () {56 vars in 1143 bytes} [Tue Aug 30 14:33:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:33:08.023005 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef6353b-3bae-406f-b888-600012d4d248 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:08.109605 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:08.119026 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef6353b-3bae-406f-b888-600012d4d248 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:08.151659 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e278b28-5bb6-4e46-b7c6-b42df1a488a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:08.152246 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e278b28-5bb6-4e46-b7c6-b42df1a488a6 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:08.152519 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e278b28-5bb6-4e46-b7c6-b42df1a488a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:08.152783 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e278b28-5bb6-4e46-b7c6-b42df1a488a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:08.217206 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e278b28-5bb6-4e46-b7c6-b42df1a488a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:08.218594 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e278b28-5bb6-4e46-b7c6-b42df1a488a6 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:08.303773 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:08.317237 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 639/1282] 10.222.0.30 () {58 vars in 1282 bytes} [Tue Aug 30 14:33:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 14:33:08.323607 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0781930b-d5ba-40da-804b-75df92371ae0 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:08.386114 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:08.423144 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0781930b-d5ba-40da-804b-75df92371ae0 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:08.456976 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb21ab33-6090-4ee5-95f9-2424819ac183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:08.457582 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21ab33-6090-4ee5-95f9-2424819ac183 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:08.457857 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21ab33-6090-4ee5-95f9-2424819ac183 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:08.458123 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21ab33-6090-4ee5-95f9-2424819ac183 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:08.467839 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb21ab33-6090-4ee5-95f9-2424819ac183 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:08.469224 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb21ab33-6090-4ee5-95f9-2424819ac183 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:08.483678 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-fb21ab33-6090-4ee5-95f9-2424819ac183 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:33:08.483678 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:33:08.483678 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:33:08.483678 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:33:08.483678 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:33:08.483678 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:33:08.483678 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:33:08.483678 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:33:08.483678 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:33:08.483678 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:33:08.483678 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:33:08.483678 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:33:08.483678 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 14:33:08.483678 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 14:33:08.483678 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 14:33:08.483678 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 14:33:08.484743 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:33:08.484743 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:33:08.484743 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:33:08.484743 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:33:08.484743 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:33:08.484743 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:33:08.484743 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:33:08.484743 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:33:08.484743 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:33:08.484743 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:33:08.484743 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:33:08.484743 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:33:08.484743 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:33:08.484743 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:33:08.484743 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:33:08.484743 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:33:08.484743 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 14:33:08.485656 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 14:33:08.485656 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 14:33:08.485656 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:33:08.485656 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 644/1283] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:33:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:33:08.491238 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e278b28-5bb6-4e46-b7c6-b42df1a488a6 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:08.586958 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e278b28-5bb6-4e46-b7c6-b42df1a488a6 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:08.619772 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0471ac19-fd47-4a4c-94d6-64849af95b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:08.620359 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471ac19-fd47-4a4c-94d6-64849af95b07 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:08.620633 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471ac19-fd47-4a4c-94d6-64849af95b07 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:08.620895 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471ac19-fd47-4a4c-94d6-64849af95b07 None admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:08.621517 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0471ac19-fd47-4a4c-94d6-64849af95b07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:33:08.621813 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0471ac19-fd47-4a4c-94d6-64849af95b07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:08.623145 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0471ac19-fd47-4a4c-94d6-64849af95b07 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:08.634690 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 640/1284] 10.222.0.30 () {56 vars in 1109 bytes} [Tue Aug 30 14:33:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:33:08.640414 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb21ab33-6090-4ee5-95f9-2424819ac183 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:08.737566 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb21ab33-6090-4ee5-95f9-2424819ac183 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:08.751135 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:08.771099 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6df82-c9b9-4e57-b6f3-1d18e53e0ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:08.771789 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6df82-c9b9-4e57-b6f3-1d18e53e0ed6 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:08.772069 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6df82-c9b9-4e57-b6f3-1d18e53e0ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:08.772337 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6df82-c9b9-4e57-b6f3-1d18e53e0ed6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:08.778609 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f6df82-c9b9-4e57-b6f3-1d18e53e0ed6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:08.780058 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f6df82-c9b9-4e57-b6f3-1d18e53e0ed6 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:08.791457 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application [None req-d0f6df82-c9b9-4e57-b6f3-1d18e53e0ed6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:33:08.791457 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:33:08.791457 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:33:08.791457 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:33:08.791457 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:33:08.791457 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:33:08.791457 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:33:08.791457 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:33:08.791457 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:33:08.791457 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:33:08.791457 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:33:08.791457 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:33:08.791457 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 14:33:08.791457 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 14:33:08.791457 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 14:33:08.792527 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 14:33:08.792527 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:33:08.792527 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:33:08.792527 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:33:08.792527 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:33:08.792527 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:33:08.792527 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:33:08.792527 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:33:08.792527 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application with Lock( Aug 30 14:33:08.792527 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:33:08.792527 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:33:08.792527 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:33:08.792527 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:33:08.792527 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:33:08.792527 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:33:08.792527 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:33:08.792527 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:33:08.793639 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 14:33:08.793639 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 14:33:08.793639 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 14:33:08.793639 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 14:33:08.793639 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 14:33:08.793639 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 14:33:08.793639 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 14:33:08.793639 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: ERROR keystone.server.flask.application Aug 30 14:33:08.793639 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 645/1285] 10.222.0.30 () {56 vars in 1111 bytes} [Tue Aug 30 14:33:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:33:08.798314 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0471ac19-fd47-4a4c-94d6-64849af95b07 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:08.846625 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:08.897110 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0471ac19-fd47-4a4c-94d6-64849af95b07 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:08.930178 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4124e-dae4-4964-992c-9edeb90b0b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:08.930801 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4124e-dae4-4964-992c-9edeb90b0b36 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:08.931081 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4124e-dae4-4964-992c-9edeb90b0b36 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:08.931383 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4124e-dae4-4964-992c-9edeb90b0b36 None admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:08.932070 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f4124e-dae4-4964-992c-9edeb90b0b36 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:33:08.932379 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f4124e-dae4-4964-992c-9edeb90b0b36 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:08.934103 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f4124e-dae4-4964-992c-9edeb90b0b36 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:08.945883 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 641/1286] 10.222.0.30 () {56 vars in 1148 bytes} [Tue Aug 30 14:33:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:33:08.951860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6df82-c9b9-4e57-b6f3-1d18e53e0ed6 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:09.049384 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6df82-c9b9-4e57-b6f3-1d18e53e0ed6 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:09.083139 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645be67f-28c9-4da9-b7f6-b97318c11eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:09.083753 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645be67f-28c9-4da9-b7f6-b97318c11eb1 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:09.084032 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645be67f-28c9-4da9-b7f6-b97318c11eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:09.084302 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645be67f-28c9-4da9-b7f6-b97318c11eb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:09.085018 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645be67f-28c9-4da9-b7f6-b97318c11eb1 None admin] RBAC: Adding query filter params (role.id=3cda0615d87d4af0860a2a5e9b4e2c05, scope.project.id=01cf71ca5abf4576b89d10c85af6d2de, user.id=dcf302a170934c25b39699088dac326a) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:33:09.085324 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645be67f-28c9-4da9-b7f6-b97318c11eb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:09.086735 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645be67f-28c9-4da9-b7f6-b97318c11eb1 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:09.092352 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 646/1287] 10.222.0.30 () {56 vars in 1374 bytes} [Tue Aug 30 14:33:08 2022] GET /identity/v3/role_assignments?user.id=dcf302a170934c25b39699088dac326a&scope.project.id=01cf71ca5abf4576b89d10c85af6d2de&role.id=3cda0615d87d4af0860a2a5e9b4e2c05 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:33:10.992792 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58660a3a-a51e-40a6-9781-1ff5db1d92c7 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:10.993379 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58660a3a-a51e-40a6-9781-1ff5db1d92c7 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:10.993379 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58660a3a-a51e-40a6-9781-1ff5db1d92c7 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:10.994691 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 642/1288] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:33:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:33:10.999767 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619621cb-585c-42c9-9d24-61e41ae5e662 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:11.000145 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619621cb-585c-42c9-9d24-61e41ae5e662 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:11.000315 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619621cb-585c-42c9-9d24-61e41ae5e662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:11.018797 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-619621cb-585c-42c9-9d24-61e41ae5e662 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:33:11.020732 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-619621cb-585c-42c9-9d24-61e41ae5e662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:33:11.049492 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:11.087849 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 647/1289] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:33:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:33:11.357285 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58660a3a-a51e-40a6-9781-1ff5db1d92c7 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:11.459928 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58660a3a-a51e-40a6-9781-1ff5db1d92c7 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:11.470147 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:11.493780 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d5235-b358-4043-8a95-09f34c49ffdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:11.494406 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d5235-b358-4043-8a95-09f34c49ffdf None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:11.494673 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d5235-b358-4043-8a95-09f34c49ffdf None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:11.494934 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d5235-b358-4043-8a95-09f34c49ffdf None admin] PATH_INFO: `/v3/services/placement` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:11.495529 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0d5235-b358-4043-8a95-09f34c49ffdf None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:11.496891 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0d5235-b358-4043-8a95-09f34c49ffdf None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:11.505993 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-fb0d5235-b358-4043-8a95-09f34c49ffdf None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 14:33:11.505993 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:33:11.505993 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:33:11.505993 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:33:11.505993 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:33:11.505993 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:33:11.505993 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:33:11.505993 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:33:11.505993 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 14:33:11.505993 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:33:11.505993 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:33:11.505993 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:33:11.505993 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:33:11.505993 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:33:11.505993 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:33:11.505993 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:33:11.505993 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:33:11.507258 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:33:11.507258 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:33:11.507258 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:33:11.507258 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:33:11.507258 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:33:11.507258 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:33:11.507258 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:33:11.507258 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:33:11.507258 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:33:11.507258 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:33:11.507258 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:33:11.507258 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:33:11.507258 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:33:11.507258 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:33:11.507258 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:33:11.507258 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:33:11.508381 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:33:11.508381 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:33:11.508381 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:33:11.508381 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:33:11.508381 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:33:11.508381 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:33:11.508381 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:33:11.508381 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:33:11.508381 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:33:11.508381 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:33:11.508381 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:33:11.508381 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:33:11.508381 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 14:33:11.508381 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:33:11.508381 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 643/1290] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Aug 30 14:33:11 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:33:11.513059 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619621cb-585c-42c9-9d24-61e41ae5e662 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:11.610510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619621cb-585c-42c9-9d24-61e41ae5e662 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:11.643724 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d579a-12b5-4eb0-8487-e67216d1f3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:11.644582 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d579a-12b5-4eb0-8487-e67216d1f3fb None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:11.644861 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d579a-12b5-4eb0-8487-e67216d1f3fb None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:11.645126 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d579a-12b5-4eb0-8487-e67216d1f3fb None admin] PATH_INFO: `/v3/services` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:11.645745 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55d579a-12b5-4eb0-8487-e67216d1f3fb None admin] RBAC: Adding query filter params (name=placement) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:33:11.646040 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55d579a-12b5-4eb0-8487-e67216d1f3fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:11.647406 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55d579a-12b5-4eb0-8487-e67216d1f3fb None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:11.652058 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 648/1291] 10.222.0.30 () {56 vars in 1116 bytes} [Tue Aug 30 14:33:11 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:33:11.657727 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d5235-b358-4043-8a95-09f34c49ffdf None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:11.756306 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d5235-b358-4043-8a95-09f34c49ffdf None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:11.788833 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70ffdd2-3854-4e32-934f-5b81f765a5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:11.789424 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ffdd2-3854-4e32-934f-5b81f765a5d9 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:11.789705 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ffdd2-3854-4e32-934f-5b81f765a5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:11.789978 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ffdd2-3854-4e32-934f-5b81f765a5d9 None admin] PATH_INFO: `/v3/services` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:11.790595 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70ffdd2-3854-4e32-934f-5b81f765a5d9 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:33:11.790896 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70ffdd2-3854-4e32-934f-5b81f765a5d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:11.792283 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70ffdd2-3854-4e32-934f-5b81f765a5d9 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:11.796573 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 644/1292] 10.222.0.30 () {56 vars in 1116 bytes} [Tue Aug 30 14:33:11 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:33:13.692082 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95254004-21d8-49fb-80f7-e16f2f57ff98 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:13.692637 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95254004-21d8-49fb-80f7-e16f2f57ff98 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:13.692637 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95254004-21d8-49fb-80f7-e16f2f57ff98 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:13.694002 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 649/1293] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:33:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:33:13.699153 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54921cb4-6bdc-4b44-b277-55cd576664f9 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:13.699590 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54921cb4-6bdc-4b44-b277-55cd576664f9 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:13.699723 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54921cb4-6bdc-4b44-b277-55cd576664f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:13.717495 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-54921cb4-6bdc-4b44-b277-55cd576664f9 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:33:13.718673 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-54921cb4-6bdc-4b44-b277-55cd576664f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:33:13.748150 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:13.784146 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 645/1294] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:33:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:33:14.057457 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95254004-21d8-49fb-80f7-e16f2f57ff98 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:14.088072 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:14.157030 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95254004-21d8-49fb-80f7-e16f2f57ff98 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:14.190657 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff030f67-10b7-47e0-9bff-075125ecec57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:14.191270 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff030f67-10b7-47e0-9bff-075125ecec57 None admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:14.191554 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff030f67-10b7-47e0-9bff-075125ecec57 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:14.191828 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff030f67-10b7-47e0-9bff-075125ecec57 None admin] PATH_INFO: `/v3/services` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:14.192468 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff030f67-10b7-47e0-9bff-075125ecec57 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:14.193815 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff030f67-10b7-47e0-9bff-075125ecec57 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:14.207587 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 650/1295] 10.222.0.30 () {60 vars in 1141 bytes} [Tue Aug 30 14:33:14 2022] POST /identity/v3/services => generated 246 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:33:16.119150 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4b98d-6337-4b24-aacc-fb5bd294e1f3 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:16.119730 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4b98d-6337-4b24-aacc-fb5bd294e1f3 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:16.119730 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4b98d-6337-4b24-aacc-fb5bd294e1f3 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:16.121049 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 646/1296] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:33:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:33:16.126174 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31160c3-abdc-4231-b3be-5c3f198ce953 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:16.126572 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31160c3-abdc-4231-b3be-5c3f198ce953 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:16.126718 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31160c3-abdc-4231-b3be-5c3f198ce953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:16.145108 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-c31160c3-abdc-4231-b3be-5c3f198ce953 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:33:16.146271 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-c31160c3-abdc-4231-b3be-5c3f198ce953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:33:16.217326 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 651/1297] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:33:16 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:33:16.497377 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a4b98d-6337-4b24-aacc-fb5bd294e1f3 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:16.595329 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a4b98d-6337-4b24-aacc-fb5bd294e1f3 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:16.627876 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a848f08f-174c-4d70-bfc7-a7a934f2270e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:16.628600 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a848f08f-174c-4d70-bfc7-a7a934f2270e None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:16.628871 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a848f08f-174c-4d70-bfc7-a7a934f2270e None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:16.629131 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a848f08f-174c-4d70-bfc7-a7a934f2270e None admin] PATH_INFO: `/v3/services/placement` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:16.629703 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a848f08f-174c-4d70-bfc7-a7a934f2270e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:16.631041 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a848f08f-174c-4d70-bfc7-a7a934f2270e None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:16.639446 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-a848f08f-174c-4d70-bfc7-a7a934f2270e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 14:33:16.639446 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:33:16.639446 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:33:16.639446 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:33:16.639446 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:33:16.639446 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:33:16.639446 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:33:16.639446 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:33:16.639446 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 14:33:16.639446 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:33:16.639446 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:33:16.639446 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:33:16.639446 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:33:16.639446 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:33:16.639446 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:33:16.639446 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:33:16.639446 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:33:16.640753 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:33:16.640753 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:33:16.640753 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:33:16.640753 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:33:16.640753 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:33:16.640753 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:33:16.640753 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:33:16.640753 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:33:16.640753 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:33:16.640753 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:33:16.640753 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:33:16.640753 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:33:16.640753 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:33:16.640753 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:33:16.640753 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:33:16.640753 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:33:16.641876 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:33:16.641876 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:33:16.641876 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:33:16.641876 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:33:16.641876 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:33:16.641876 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:33:16.641876 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:33:16.641876 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:33:16.641876 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:33:16.641876 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:33:16.641876 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:33:16.641876 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:33:16.641876 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 14:33:16.641876 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:33:16.641876 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 647/1298] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Aug 30 14:33:16 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:33:16.647007 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31160c3-abdc-4231-b3be-5c3f198ce953 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:16.666057 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:16.745113 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31160c3-abdc-4231-b3be-5c3f198ce953 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:16.777863 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8723f969-187b-4b0f-9e7c-60f7dccd4687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:16.778406 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8723f969-187b-4b0f-9e7c-60f7dccd4687 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:16.778683 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8723f969-187b-4b0f-9e7c-60f7dccd4687 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:16.778953 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8723f969-187b-4b0f-9e7c-60f7dccd4687 None admin] PATH_INFO: `/v3/services` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:16.779604 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8723f969-187b-4b0f-9e7c-60f7dccd4687 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:33:16.779904 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8723f969-187b-4b0f-9e7c-60f7dccd4687 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:16.781276 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8723f969-187b-4b0f-9e7c-60f7dccd4687 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:16.786895 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 652/1299] 10.222.0.30 () {56 vars in 1116 bytes} [Tue Aug 30 14:33:16 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:33:16.792707 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a848f08f-174c-4d70-bfc7-a7a934f2270e None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:16.823343 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:16.890048 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a848f08f-174c-4d70-bfc7-a7a934f2270e None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:16.922991 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc05116a-edb3-4862-9bd2-b0f4fa8c1625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:16.923614 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc05116a-edb3-4862-9bd2-b0f4fa8c1625 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:16.923893 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc05116a-edb3-4862-9bd2-b0f4fa8c1625 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:16.924163 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc05116a-edb3-4862-9bd2-b0f4fa8c1625 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:16.924872 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc05116a-edb3-4862-9bd2-b0f4fa8c1625 None admin] RBAC: Adding query filter params (interface=public, service_id=2fff9eb63458461c8f0f78f4e54a1031, region_id=RegionOne) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:33:16.925178 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc05116a-edb3-4862-9bd2-b0f4fa8c1625 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:16.926624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc05116a-edb3-4862-9bd2-b0f4fa8c1625 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:16.931460 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 648/1300] 10.222.0.30 () {56 vars in 1251 bytes} [Tue Aug 30 14:33:16 2022] GET /identity/v3/endpoints?service_id=2fff9eb63458461c8f0f78f4e54a1031&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:33:16.937547 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8723f969-187b-4b0f-9e7c-60f7dccd4687 None admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:17.019444 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:17.037941 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8723f969-187b-4b0f-9e7c-60f7dccd4687 None admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:17.071292 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da366c62-dfa3-4bf9-87bd-03d15835003b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:17.071884 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da366c62-dfa3-4bf9-87bd-03d15835003b None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:17.072159 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da366c62-dfa3-4bf9-87bd-03d15835003b None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:17.072455 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da366c62-dfa3-4bf9-87bd-03d15835003b None admin] PATH_INFO: `/v3/services` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:17.073046 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da366c62-dfa3-4bf9-87bd-03d15835003b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:17.074396 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da366c62-dfa3-4bf9-87bd-03d15835003b None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:17.080959 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 653/1301] 10.222.0.30 () {56 vars in 1087 bytes} [Tue Aug 30 14:33:16 2022] GET /identity/v3/services => generated 2191 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:33:18.970115 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9eb10-c434-40bb-a928-481b9650b05b None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:18.970649 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9eb10-c434-40bb-a928-481b9650b05b None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:18.970649 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9eb10-c434-40bb-a928-481b9650b05b None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:18.972035 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 649/1302] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:33:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:33:18.977034 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284528ea-4d57-4066-a75f-9205779bbd2c None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:18.977445 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284528ea-4d57-4066-a75f-9205779bbd2c None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:18.977581 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284528ea-4d57-4066-a75f-9205779bbd2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:18.995899 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-284528ea-4d57-4066-a75f-9205779bbd2c None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:33:18.997078 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-284528ea-4d57-4066-a75f-9205779bbd2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:33:19.062306 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 654/1303] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:33:18 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 14:33:19.337003 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9eb10-c434-40bb-a928-481b9650b05b None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:19.434933 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:19.442497 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9eb10-c434-40bb-a928-481b9650b05b None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:19.474695 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df10586-688f-4215-be44-db7e8d71f4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:19.475338 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df10586-688f-4215-be44-db7e8d71f4d6 None admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:19.475614 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df10586-688f-4215-be44-db7e8d71f4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:19.475881 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df10586-688f-4215-be44-db7e8d71f4d6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:19.476447 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df10586-688f-4215-be44-db7e8d71f4d6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:19.477794 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df10586-688f-4215-be44-db7e8d71f4d6 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:19.486176 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application [None req-8df10586-688f-4215-be44-db7e8d71f4d6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 14:33:19.486176 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:33:19.486176 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 14:33:19.486176 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 14:33:19.486176 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 14:33:19.486176 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 14:33:19.486176 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 14:33:19.486176 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:33:19.486176 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 14:33:19.486176 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:33:19.486176 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 14:33:19.486176 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:33:19.486176 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 14:33:19.486176 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 14:33:19.486176 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 14:33:19.486176 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 14:33:19.486176 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 14:33:19.487199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 14:33:19.487199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 14:33:19.487199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 14:33:19.487199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 14:33:19.487199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 14:33:19.487199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 14:33:19.487199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 14:33:19.487199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 14:33:19.487199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 14:33:19.487199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 14:33:19.487199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 14:33:19.487199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 14:33:19.487199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 14:33:19.487199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 14:33:19.487199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 14:33:19.487199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 14:33:19.488771 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 14:33:19.488771 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application with Lock( Aug 30 14:33:19.488771 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 14:33:19.488771 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self._enter() Aug 30 14:33:19.488771 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 14:33:19.488771 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 14:33:19.488771 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 14:33:19.488771 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application return self.creator() Aug 30 14:33:19.488771 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 14:33:19.488771 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application created_value = creator( Aug 30 14:33:19.488771 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 14:33:19.488771 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 14:33:19.488771 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 14:33:19.488771 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: ERROR keystone.server.flask.application Aug 30 14:33:19.488771 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 650/1304] 10.222.0.30 () {56 vars in 1117 bytes} [Tue Aug 30 14:33:19 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 14:33:19.494228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284528ea-4d57-4066-a75f-9205779bbd2c None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:19.568924 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:19.592177 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284528ea-4d57-4066-a75f-9205779bbd2c None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:19.625248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d10b2f2-5085-47bb-a1b0-7bca733fd4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:19.625850 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d10b2f2-5085-47bb-a1b0-7bca733fd4c6 None admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:19.626128 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d10b2f2-5085-47bb-a1b0-7bca733fd4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:19.626395 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d10b2f2-5085-47bb-a1b0-7bca733fd4c6 None admin] PATH_INFO: `/v3/services` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:19.627012 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d10b2f2-5085-47bb-a1b0-7bca733fd4c6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:33:19.627339 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d10b2f2-5085-47bb-a1b0-7bca733fd4c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:19.628827 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d10b2f2-5085-47bb-a1b0-7bca733fd4c6 None admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:19.633795 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 655/1305] 10.222.0.30 () {56 vars in 1116 bytes} [Tue Aug 30 14:33:19 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:33:19.639575 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df10586-688f-4215-be44-db7e8d71f4d6 None admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:33:19.738336 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df10586-688f-4215-be44-db7e8d71f4d6 None admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:33:19.771007 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a4584b-1ba1-4cf1-a207-14fa57378a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:33:19.771682 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4584b-1ba1-4cf1-a207-14fa57378a79 None admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:33:19.771958 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4584b-1ba1-4cf1-a207-14fa57378a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:33:19.772223 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4584b-1ba1-4cf1-a207-14fa57378a79 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:33:19.772866 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a4584b-1ba1-4cf1-a207-14fa57378a79 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:33:19.774203 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a4584b-1ba1-4cf1-a207-14fa57378a79 None admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:33:19.793067 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 651/1306] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:33:19 2022] POST /identity/v3/endpoints => generated 331 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:33:29.630863 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=30 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:33:29.788944 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:23.380452 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601a928-1851-4fc9-826b-7c49ed651a4c None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:23.381534 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601a928-1851-4fc9-826b-7c49ed651a4c None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:23.382091 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601a928-1851-4fc9-826b-7c49ed651a4c None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:23.389140 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5389205-f9d0-4e09-8089-94e5fc69d119 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:23.389994 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5389205-f9d0-4e09-8089-94e5fc69d119 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:23.390498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5389205-f9d0-4e09-8089-94e5fc69d119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:23.393646 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 656/1307] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:34:23 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:34:23.480607 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-a5389205-f9d0-4e09-8089-94e5fc69d119 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:34:23.481735 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-a5389205-f9d0-4e09-8089-94e5fc69d119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:23.616005 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 652/1308] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:34:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:34:23.930871 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea49dc0-2c54-4c3d-82c3-0f496e10c717 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:23.931354 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea49dc0-2c54-4c3d-82c3-0f496e10c717 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:23.931641 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea49dc0-2c54-4c3d-82c3-0f496e10c717 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:23.933972 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 657/1309] 10.222.0.30 () {54 vars in 965 bytes} [Tue Aug 30 14:34:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:34:23.939668 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c828e6a-f7a9-4bf3-b553-e26e16cfcccf None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:23.940080 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c828e6a-f7a9-4bf3-b553-e26e16cfcccf None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:23.940504 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c828e6a-f7a9-4bf3-b553-e26e16cfcccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:24.022565 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-4c828e6a-f7a9-4bf3-b553-e26e16cfcccf None None] MFA Rules not processed for user `9e14e86e085d4987aece69fe64867ce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:34:24.023891 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-4c828e6a-f7a9-4bf3-b553-e26e16cfcccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:24.046090 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:24.135117 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea49dc0-2c54-4c3d-82c3-0f496e10c717 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:24.136968 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-cea49dc0-2c54-4c3d-82c3-0f496e10c717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:24.141155 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 653/1310] 10.222.0.30 () {58 vars in 1062 bytes} [Tue Aug 30 14:34:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:34:24.268045 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea49dc0-2c54-4c3d-82c3-0f496e10c717 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:24.357576 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba394359-9bca-49d5-848f-a75ee661ef5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:24.358218 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba394359-9bca-49d5-848f-a75ee661ef5a service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:24.358495 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba394359-9bca-49d5-848f-a75ee661ef5a service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:24.358762 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba394359-9bca-49d5-848f-a75ee661ef5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:24.360396 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-ba394359-9bca-49d5-848f-a75ee661ef5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:24.421667 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:24.485878 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba394359-9bca-49d5-848f-a75ee661ef5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:24.488294 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba394359-9bca-49d5-848f-a75ee661ef5a service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:24.524088 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 658/1311] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:34:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:34:26.762101 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a8fdf-c7bd-4c1b-abd7-1a45cb5778b3 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:26.762673 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a8fdf-c7bd-4c1b-abd7-1a45cb5778b3 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:26.762673 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a8fdf-c7bd-4c1b-abd7-1a45cb5778b3 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:26.764079 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 654/1312] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:34:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:34:26.769876 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806fe0a9-6529-4028-b8db-24a252792f4c None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:26.770410 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806fe0a9-6529-4028-b8db-24a252792f4c None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:26.770619 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806fe0a9-6529-4028-b8db-24a252792f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:26.849357 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-806fe0a9-6529-4028-b8db-24a252792f4c None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:34:26.850556 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-806fe0a9-6529-4028-b8db-24a252792f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:26.944473 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 659/1313] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:34:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:34:27.224309 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1a8fdf-c7bd-4c1b-abd7-1a45cb5778b3 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:27.225844 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-ee1a8fdf-c7bd-4c1b-abd7-1a45cb5778b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:27.328325 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1a8fdf-c7bd-4c1b-abd7-1a45cb5778b3 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:27.397735 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:27.400137 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d9dcf-1fc6-4877-a49c-b749f325c53a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:27.400777 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d9dcf-1fc6-4877-a49c-b749f325c53a service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:27.401248 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d9dcf-1fc6-4877-a49c-b749f325c53a service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:27.401684 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d9dcf-1fc6-4877-a49c-b749f325c53a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:27.403525 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-402d9dcf-1fc6-4877-a49c-b749f325c53a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:27.510638 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402d9dcf-1fc6-4877-a49c-b749f325c53a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:27.512104 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402d9dcf-1fc6-4877-a49c-b749f325c53a service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:27.530442 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 655/1314] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:34:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:34:29.733465 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43024de9-aec8-445f-b15a-eb389edd0334 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:29.734011 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43024de9-aec8-445f-b15a-eb389edd0334 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:29.734011 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43024de9-aec8-445f-b15a-eb389edd0334 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:29.735386 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 660/1315] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:34:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:34:29.740564 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da262c-e414-44ff-9b41-7c1857f7c20b None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:29.740993 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da262c-e414-44ff-9b41-7c1857f7c20b None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:29.741118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da262c-e414-44ff-9b41-7c1857f7c20b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:29.805877 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-40da262c-e414-44ff-9b41-7c1857f7c20b None None] MFA Rules not processed for user `873ad929e6dd4a1c9f5875c6017d0fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:34:29.807095 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-40da262c-e414-44ff-9b41-7c1857f7c20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:29.859813 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:29.909260 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 656/1316] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:34:29 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:34:30.194660 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43024de9-aec8-445f-b15a-eb389edd0334 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:30.196242 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-43024de9-aec8-445f-b15a-eb389edd0334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:30.206350 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:30.302368 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43024de9-aec8-445f-b15a-eb389edd0334 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:30.375538 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f5ab5-21a8-464d-9e28-395991276b17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:30.376131 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f5ab5-21a8-464d-9e28-395991276b17 service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:30.376404 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f5ab5-21a8-464d-9e28-395991276b17 service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:30.377365 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f5ab5-21a8-464d-9e28-395991276b17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:30.378929 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-428f5ab5-21a8-464d-9e28-395991276b17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:30.441141 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:30.487773 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428f5ab5-21a8-464d-9e28-395991276b17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:30.489218 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428f5ab5-21a8-464d-9e28-395991276b17 service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:30.507347 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 661/1317] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:34:30 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:34:32.961539 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29538664-330a-46b7-acbc-133cc54ad849 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:32.962075 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29538664-330a-46b7-acbc-133cc54ad849 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:32.962075 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29538664-330a-46b7-acbc-133cc54ad849 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:32.963541 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 657/1318] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:34:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:34:32.968794 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6478889-d601-4839-b0fe-e2807cb148be None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:32.969219 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6478889-d601-4839-b0fe-e2807cb148be None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:32.969316 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6478889-d601-4839-b0fe-e2807cb148be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:33.034828 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-c6478889-d601-4839-b0fe-e2807cb148be None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:34:33.036040 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-c6478889-d601-4839-b0fe-e2807cb148be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:33.135448 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 662/1319] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:34:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:34:33.415678 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29538664-330a-46b7-acbc-133cc54ad849 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:33.417216 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-29538664-330a-46b7-acbc-133cc54ad849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:33.517322 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29538664-330a-46b7-acbc-133cc54ad849 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:33.590138 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf32a19-f735-467f-b2ea-e70082f58103 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:33.590731 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf32a19-f735-467f-b2ea-e70082f58103 service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:33.591006 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf32a19-f735-467f-b2ea-e70082f58103 service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:33.591300 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf32a19-f735-467f-b2ea-e70082f58103 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:33.592875 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-dcf32a19-f735-467f-b2ea-e70082f58103 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:33.600606 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:33.699854 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf32a19-f735-467f-b2ea-e70082f58103 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:33.701202 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf32a19-f735-467f-b2ea-e70082f58103 service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:33.718853 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 658/1320] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:34:33 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:34:36.274474 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb06f3b-f713-4522-855b-5b40f105d59e None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:36.275047 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb06f3b-f713-4522-855b-5b40f105d59e None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:36.275047 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb06f3b-f713-4522-855b-5b40f105d59e None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:36.276540 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 663/1321] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:34:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:34:36.281545 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feebb8cb-9e9b-4a08-b826-b7618b1adb07 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:36.281969 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feebb8cb-9e9b-4a08-b826-b7618b1adb07 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:36.282112 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feebb8cb-9e9b-4a08-b826-b7618b1adb07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:36.350452 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-feebb8cb-9e9b-4a08-b826-b7618b1adb07 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:34:36.351707 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-feebb8cb-9e9b-4a08-b826-b7618b1adb07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:36.409446 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:36.448961 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 659/1322] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:34:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:34:36.734298 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb06f3b-f713-4522-855b-5b40f105d59e None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:36.735963 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-dfb06f3b-f713-4522-855b-5b40f105d59e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:36.755458 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:36.839747 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb06f3b-f713-4522-855b-5b40f105d59e None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:36.912926 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034fe006-7e2d-42d1-9f6c-814674306f00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:36.913423 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034fe006-7e2d-42d1-9f6c-814674306f00 service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:36.913703 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034fe006-7e2d-42d1-9f6c-814674306f00 service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:36.913969 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034fe006-7e2d-42d1-9f6c-814674306f00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:36.915590 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-034fe006-7e2d-42d1-9f6c-814674306f00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:36.987111 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:37.025786 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034fe006-7e2d-42d1-9f6c-814674306f00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:37.027233 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034fe006-7e2d-42d1-9f6c-814674306f00 service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:37.045315 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 664/1323] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:34:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:34:39.010027 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0adbe-f5d8-4a4b-a5d3-fcd18173c289 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:39.010597 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0adbe-f5d8-4a4b-a5d3-fcd18173c289 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:39.010597 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb0adbe-f5d8-4a4b-a5d3-fcd18173c289 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:39.011849 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 660/1324] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:34:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:34:39.017492 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7cf165-0e9f-417a-92fd-54cb5d8f7759 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:39.017950 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7cf165-0e9f-417a-92fd-54cb5d8f7759 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:39.018212 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7cf165-0e9f-417a-92fd-54cb5d8f7759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:39.086706 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-3c7cf165-0e9f-417a-92fd-54cb5d8f7759 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:34:39.087965 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-3c7cf165-0e9f-417a-92fd-54cb5d8f7759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:39.186102 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 665/1325] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:34:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:34:39.467943 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb0adbe-f5d8-4a4b-a5d3-fcd18173c289 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:39.469502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-7cb0adbe-f5d8-4a4b-a5d3-fcd18173c289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:39.584419 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb0adbe-f5d8-4a4b-a5d3-fcd18173c289 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:39.660068 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34330885-fce9-4a58-ad6c-e3dfd07cdc9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:39.660684 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34330885-fce9-4a58-ad6c-e3dfd07cdc9f service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:39.660963 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34330885-fce9-4a58-ad6c-e3dfd07cdc9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:39.661251 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34330885-fce9-4a58-ad6c-e3dfd07cdc9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:39.662842 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-34330885-fce9-4a58-ad6c-e3dfd07cdc9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:39.679455 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:39.772712 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34330885-fce9-4a58-ad6c-e3dfd07cdc9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:39.774138 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34330885-fce9-4a58-ad6c-e3dfd07cdc9f service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:39.791775 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 661/1326] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:34:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:34:41.768165 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe5f2b-0d20-4ba4-8561-be5f6c16ecef None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:41.768747 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe5f2b-0d20-4ba4-8561-be5f6c16ecef None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:41.768747 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe5f2b-0d20-4ba4-8561-be5f6c16ecef None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:41.770147 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 666/1327] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:34:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:34:41.776075 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f72ba60-3c31-4fef-93dd-928c13469449 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:41.776657 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f72ba60-3c31-4fef-93dd-928c13469449 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:41.776657 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f72ba60-3c31-4fef-93dd-928c13469449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:41.845886 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-3f72ba60-3c31-4fef-93dd-928c13469449 None None] MFA Rules not processed for user `873ad929e6dd4a1c9f5875c6017d0fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:34:41.847088 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-3f72ba60-3c31-4fef-93dd-928c13469449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:41.913338 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:41.950935 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 662/1328] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:34:41 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:34:42.237601 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe5f2b-0d20-4ba4-8561-be5f6c16ecef None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:42.239160 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-d1fe5f2b-0d20-4ba4-8561-be5f6c16ecef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:42.275052 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:42.348306 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe5f2b-0d20-4ba4-8561-be5f6c16ecef None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:42.421345 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5d2437-3305-4730-b745-8c52723424fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:42.421872 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d2437-3305-4730-b745-8c52723424fa service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:42.422145 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d2437-3305-4730-b745-8c52723424fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:42.422412 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d2437-3305-4730-b745-8c52723424fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:42.424801 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-fb5d2437-3305-4730-b745-8c52723424fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:42.503343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:42.538678 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5d2437-3305-4730-b745-8c52723424fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:42.540161 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5d2437-3305-4730-b745-8c52723424fa service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:42.559301 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 667/1329] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:34:42 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:34:44.682713 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7c782a-6140-4b68-bcc8-363886ea4f73 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:44.683275 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7c782a-6140-4b68-bcc8-363886ea4f73 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:44.683645 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7c782a-6140-4b68-bcc8-363886ea4f73 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:44.684914 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 663/1330] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:34:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:34:44.690348 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06139570-a500-4806-8924-ee24e8cce2af None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:44.690782 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06139570-a500-4806-8924-ee24e8cce2af None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:44.690892 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06139570-a500-4806-8924-ee24e8cce2af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:44.756245 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-06139570-a500-4806-8924-ee24e8cce2af None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:34:44.757452 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-06139570-a500-4806-8924-ee24e8cce2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:44.852467 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 668/1331] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:34:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:34:45.134763 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7c782a-6140-4b68-bcc8-363886ea4f73 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:45.136329 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-ed7c782a-6140-4b68-bcc8-363886ea4f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:45.248250 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7c782a-6140-4b68-bcc8-363886ea4f73 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:45.320980 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639cfafd-a130-4ead-ad41-52c6a86cd569 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:45.321540 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639cfafd-a130-4ead-ad41-52c6a86cd569 service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:45.321935 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639cfafd-a130-4ead-ad41-52c6a86cd569 service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:45.322234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639cfafd-a130-4ead-ad41-52c6a86cd569 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:45.324237 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-639cfafd-a130-4ead-ad41-52c6a86cd569 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:45.352373 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:45.432219 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639cfafd-a130-4ead-ad41-52c6a86cd569 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:45.433894 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639cfafd-a130-4ead-ad41-52c6a86cd569 service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:45.451812 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 664/1332] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:34:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:34:47.410408 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5270365-0cd1-4bd8-9e11-1a469689f64e None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:47.410951 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5270365-0cd1-4bd8-9e11-1a469689f64e None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:47.410951 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5270365-0cd1-4bd8-9e11-1a469689f64e None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:47.412243 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 669/1333] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:34:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:34:47.417338 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d27ec-5386-47f0-83ec-bb30025ad751 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:47.417777 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d27ec-5386-47f0-83ec-bb30025ad751 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:47.417879 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d27ec-5386-47f0-83ec-bb30025ad751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:47.483703 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-044d27ec-5386-47f0-83ec-bb30025ad751 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:34:47.484927 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-044d27ec-5386-47f0-83ec-bb30025ad751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:47.561679 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:47.585110 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 665/1334] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:34:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:34:47.874450 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5270365-0cd1-4bd8-9e11-1a469689f64e None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:47.876116 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-d5270365-0cd1-4bd8-9e11-1a469689f64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:47.915999 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:47.981455 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5270365-0cd1-4bd8-9e11-1a469689f64e None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:48.056692 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb72ea-05ea-48e1-914b-dfdbe9387409 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:48.057287 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb72ea-05ea-48e1-914b-dfdbe9387409 service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:48.057569 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb72ea-05ea-48e1-914b-dfdbe9387409 service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:48.057837 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb72ea-05ea-48e1-914b-dfdbe9387409 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:48.059692 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-3bbb72ea-05ea-48e1-914b-dfdbe9387409 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:48.142030 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:48.167698 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbb72ea-05ea-48e1-914b-dfdbe9387409 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:48.169130 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbb72ea-05ea-48e1-914b-dfdbe9387409 service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:48.186910 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 670/1335] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:34:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:34:50.402041 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8772885-cbce-4de3-89e0-633b3d4ff7d0 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:50.402598 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8772885-cbce-4de3-89e0-633b3d4ff7d0 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:50.402598 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8772885-cbce-4de3-89e0-633b3d4ff7d0 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:50.403970 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 666/1336] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:34:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:34:50.409174 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6467ae-927b-4154-ab03-7238374e8956 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:50.409574 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6467ae-927b-4154-ab03-7238374e8956 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:50.409731 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6467ae-927b-4154-ab03-7238374e8956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:50.475687 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-ac6467ae-927b-4154-ab03-7238374e8956 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:34:50.476821 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-ac6467ae-927b-4154-ab03-7238374e8956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:50.572267 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 671/1337] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:34:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:34:50.850788 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8772885-cbce-4de3-89e0-633b3d4ff7d0 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:50.852380 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-d8772885-cbce-4de3-89e0-633b3d4ff7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:50.954874 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8772885-cbce-4de3-89e0-633b3d4ff7d0 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:51.026285 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d255b6a2-992c-446d-a90f-6093cecbfb5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:51.026883 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255b6a2-992c-446d-a90f-6093cecbfb5f service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:51.027162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255b6a2-992c-446d-a90f-6093cecbfb5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:51.027459 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255b6a2-992c-446d-a90f-6093cecbfb5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:51.029042 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-d255b6a2-992c-446d-a90f-6093cecbfb5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:51.065029 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:51.138626 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d255b6a2-992c-446d-a90f-6093cecbfb5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:51.140048 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d255b6a2-992c-446d-a90f-6093cecbfb5f service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:51.158186 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 667/1338] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:34:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:34:54.957652 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0b2a7-2ed9-4adc-8fd2-312c0a2dd677 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:54.958159 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0b2a7-2ed9-4adc-8fd2-312c0a2dd677 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:54.958251 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0b2a7-2ed9-4adc-8fd2-312c0a2dd677 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:54.959739 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 672/1339] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:34:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:34:54.964860 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4341f5c-1344-47b0-adaa-f3a1c3b3cf68 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:54.965296 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4341f5c-1344-47b0-adaa-f3a1c3b3cf68 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:54.965392 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4341f5c-1344-47b0-adaa-f3a1c3b3cf68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:55.030557 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-b4341f5c-1344-47b0-adaa-f3a1c3b3cf68 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:34:55.032414 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-b4341f5c-1344-47b0-adaa-f3a1c3b3cf68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:55.118174 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:55.130101 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 668/1340] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:34:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:34:55.431307 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8537c-50b3-476f-a09e-e7c18c59e804 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:55.431601 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8537c-50b3-476f-a09e-e7c18c59e804 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:55.431879 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8537c-50b3-476f-a09e-e7c18c59e804 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:55.433250 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 673/1341] 10.222.0.30 () {54 vars in 965 bytes} [Tue Aug 30 14:34:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:34:55.440044 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ddc51e-6692-4f57-b5b0-e94326dd6638 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:55.440331 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ddc51e-6692-4f57-b5b0-e94326dd6638 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:55.440604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ddc51e-6692-4f57-b5b0-e94326dd6638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:55.518422 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-68ddc51e-6692-4f57-b5b0-e94326dd6638 None None] MFA Rules not processed for user `9e14e86e085d4987aece69fe64867ce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:34:55.519676 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-68ddc51e-6692-4f57-b5b0-e94326dd6638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:55.596761 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 669/1342] 10.222.0.30 () {58 vars in 1062 bytes} [Tue Aug 30 14:34:55 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:34:55.605572 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b8537c-50b3-476f-a09e-e7c18c59e804 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:55.607083 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-32b8537c-50b3-476f-a09e-e7c18c59e804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:55.655773 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:55.713112 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b8537c-50b3-476f-a09e-e7c18c59e804 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:55.787083 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f254c-a223-4bf3-b7f5-96514689a026 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:55.787706 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f254c-a223-4bf3-b7f5-96514689a026 service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:55.787978 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f254c-a223-4bf3-b7f5-96514689a026 service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:55.788242 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f254c-a223-4bf3-b7f5-96514689a026 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:55.789817 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-8e6f254c-a223-4bf3-b7f5-96514689a026 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:55.881301 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:55.898498 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6f254c-a223-4bf3-b7f5-96514689a026 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:55.899960 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6f254c-a223-4bf3-b7f5-96514689a026 service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:55.918596 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 674/1343] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:34:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:34:59.110724 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a04bf-3af8-45b9-adea-afd05e0e17a8 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:59.111291 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a04bf-3af8-45b9-adea-afd05e0e17a8 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:59.111291 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a04bf-3af8-45b9-adea-afd05e0e17a8 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:59.112652 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 670/1344] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:34:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:34:59.117802 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f194a87-b6dd-442e-9999-170f96865d47 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:59.118210 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f194a87-b6dd-442e-9999-170f96865d47 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:59.118466 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f194a87-b6dd-442e-9999-170f96865d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:59.185823 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-8f194a87-b6dd-442e-9999-170f96865d47 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:34:59.187043 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-8f194a87-b6dd-442e-9999-170f96865d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:59.288545 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 675/1345] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:34:59 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:34:59.576259 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a04bf-3af8-45b9-adea-afd05e0e17a8 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:34:59.577792 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-858a04bf-3af8-45b9-adea-afd05e0e17a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:59.667957 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:34:59.686489 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a04bf-3af8-45b9-adea-afd05e0e17a8 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:34:59.761169 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f270efa-8b9e-4fe8-9043-1c6213d51979 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:34:59.761808 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f270efa-8b9e-4fe8-9043-1c6213d51979 service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:34:59.762120 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f270efa-8b9e-4fe8-9043-1c6213d51979 service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:34:59.762595 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f270efa-8b9e-4fe8-9043-1c6213d51979 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:34:59.764494 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-3f270efa-8b9e-4fe8-9043-1c6213d51979 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:34:59.895758 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f270efa-8b9e-4fe8-9043-1c6213d51979 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:34:59.897403 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f270efa-8b9e-4fe8-9043-1c6213d51979 service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:34:59.917252 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 671/1346] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:34:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:35:03.865123 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754b809-8eb9-46fc-b00a-06434df03079 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:03.865658 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754b809-8eb9-46fc-b00a-06434df03079 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:03.865658 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754b809-8eb9-46fc-b00a-06434df03079 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:03.867005 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 676/1347] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:35:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:35:03.872575 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca9ed5-dc0c-41af-977a-21d2d0c0f7ab None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:03.872980 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca9ed5-dc0c-41af-977a-21d2d0c0f7ab None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:03.873129 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca9ed5-dc0c-41af-977a-21d2d0c0f7ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:03.889413 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:35:03.944530 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-eeca9ed5-dc0c-41af-977a-21d2d0c0f7ab None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:35:03.945666 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-eeca9ed5-dc0c-41af-977a-21d2d0c0f7ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:04.050877 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 672/1348] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:35:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:35:04.337653 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8754b809-8eb9-46fc-b00a-06434df03079 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:35:04.339302 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-8754b809-8eb9-46fc-b00a-06434df03079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:04.397163 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:35:04.444053 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8754b809-8eb9-46fc-b00a-06434df03079 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:35:04.516617 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e7a196-2b95-4db5-9efd-ac892188dad2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:35:04.517225 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e7a196-2b95-4db5-9efd-ac892188dad2 service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:04.517503 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e7a196-2b95-4db5-9efd-ac892188dad2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:04.517773 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e7a196-2b95-4db5-9efd-ac892188dad2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:04.519391 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-19e7a196-2b95-4db5-9efd-ac892188dad2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:04.623521 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:35:04.627971 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e7a196-2b95-4db5-9efd-ac892188dad2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:35:04.629402 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e7a196-2b95-4db5-9efd-ac892188dad2 service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:35:04.648059 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 677/1349] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:35:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:35:09.567025 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a32786-5497-485d-afd6-2bd9ee0f87ac None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:09.567604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a32786-5497-485d-afd6-2bd9ee0f87ac None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:09.567604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a32786-5497-485d-afd6-2bd9ee0f87ac None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:09.569166 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 673/1350] 10.222.0.30 () {54 vars in 921 bytes} [Tue Aug 30 14:35:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:35:09.574063 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1bec8-88d6-41bb-9a9f-ef6ff191883b None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:09.574494 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1bec8-88d6-41bb-9a9f-ef6ff191883b None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:09.574926 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1bec8-88d6-41bb-9a9f-ef6ff191883b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:09.675403 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-64b1bec8-88d6-41bb-9a9f-ef6ff191883b None None] MFA Rules not processed for user `dcf302a170934c25b39699088dac326a`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:35:09.676860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-64b1bec8-88d6-41bb-9a9f-ef6ff191883b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:09.863733 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 678/1351] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Aug 30 14:35:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:35:13.889556 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f165e-b1e4-4f77-bfe0-980a760dd11c None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:13.890940 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f165e-b1e4-4f77-bfe0-980a760dd11c None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:13.891423 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f165e-b1e4-4f77-bfe0-980a760dd11c None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:13.892939 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 674/1352] 10.222.0.30 () {54 vars in 921 bytes} [Tue Aug 30 14:35:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:35:13.898136 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d9db5b-1e53-4028-8c35-d5b40cf80375 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:13.898563 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d9db5b-1e53-4028-8c35-d5b40cf80375 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:13.898885 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d9db5b-1e53-4028-8c35-d5b40cf80375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:13.957252 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:35:13.982300 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-42d9db5b-1e53-4028-8c35-d5b40cf80375 None None] MFA Rules not processed for user `dcf302a170934c25b39699088dac326a`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:35:13.983702 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-42d9db5b-1e53-4028-8c35-d5b40cf80375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:14.049002 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=59 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:35:14.101018 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 679/1353] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Aug 30 14:35:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:35:15.598862 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6f48f3-6475-4ae2-b529-5206bb61c537 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:15.599433 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6f48f3-6475-4ae2-b529-5206bb61c537 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:15.599433 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6f48f3-6475-4ae2-b529-5206bb61c537 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:15.600765 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 675/1354] 10.222.0.30 () {54 vars in 921 bytes} [Tue Aug 30 14:35:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:35:15.606271 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa7b6db-7535-4421-8661-bb4e3eb1f7d1 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:15.606679 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa7b6db-7535-4421-8661-bb4e3eb1f7d1 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:15.606820 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa7b6db-7535-4421-8661-bb4e3eb1f7d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:15.685532 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-dfa7b6db-7535-4421-8661-bb4e3eb1f7d1 None None] MFA Rules not processed for user `dcf302a170934c25b39699088dac326a`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:35:15.686736 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-dfa7b6db-7535-4421-8661-bb4e3eb1f7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:15.728612 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:35:15.794626 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 680/1355] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Aug 30 14:35:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:35:19.178965 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04450b31-5879-48f5-9cc6-34706588baa8 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:19.179482 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04450b31-5879-48f5-9cc6-34706588baa8 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:19.179581 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04450b31-5879-48f5-9cc6-34706588baa8 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:19.180873 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 676/1356] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:35:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:35:19.186234 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6433c4d0-84dd-4b72-ac30-4fe4a937ac1e None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:19.186638 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6433c4d0-84dd-4b72-ac30-4fe4a937ac1e None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:19.186780 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6433c4d0-84dd-4b72-ac30-4fe4a937ac1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:19.284930 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-6433c4d0-84dd-4b72-ac30-4fe4a937ac1e None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:35:19.286460 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-6433c4d0-84dd-4b72-ac30-4fe4a937ac1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:19.430287 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 681/1357] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:35:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:35:19.740159 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2d5a4-fa85-4f65-97f9-f12ce16d28a0 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:19.740921 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2d5a4-fa85-4f65-97f9-f12ce16d28a0 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:19.741576 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2d5a4-fa85-4f65-97f9-f12ce16d28a0 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:19.744748 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 677/1358] 10.222.0.30 () {54 vars in 962 bytes} [Tue Aug 30 14:35:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:35:19.753597 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4081c3-3a78-42c1-bf55-dc8fa01c9fd7 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:19.754385 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4081c3-3a78-42c1-bf55-dc8fa01c9fd7 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:19.754952 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4081c3-3a78-42c1-bf55-dc8fa01c9fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:19.775607 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:35:19.840098 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-3b4081c3-3a78-42c1-bf55-dc8fa01c9fd7 None None] MFA Rules not processed for user `8878de66a03044e38e507028853a0cd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:35:19.841303 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-3b4081c3-3a78-42c1-bf55-dc8fa01c9fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:19.963805 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 682/1359] 10.222.0.30 () {58 vars in 1059 bytes} [Tue Aug 30 14:35:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:35:19.972184 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2d5a4-fa85-4f65-97f9-f12ce16d28a0 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:35:19.973681 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-2af2d5a4-fa85-4f65-97f9-f12ce16d28a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:20.132976 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2d5a4-fa85-4f65-97f9-f12ce16d28a0 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:35:20.217892 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:35:20.263163 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e426abcc-fe54-465d-847d-7f117e55f82f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:35:20.263715 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e426abcc-fe54-465d-847d-7f117e55f82f service nova] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:20.263988 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e426abcc-fe54-465d-847d-7f117e55f82f service nova] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:20.264250 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e426abcc-fe54-465d-847d-7f117e55f82f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:20.266569 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-e426abcc-fe54-465d-847d-7f117e55f82f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:20.368409 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e426abcc-fe54-465d-847d-7f117e55f82f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:35:20.370011 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e426abcc-fe54-465d-847d-7f117e55f82f service nova] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:35:20.391175 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 678/1360] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:35:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:35:22.536532 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f0371-ba34-4aae-a84e-e1535054b574 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:22.537090 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f0371-ba34-4aae-a84e-e1535054b574 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:22.537090 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f0371-ba34-4aae-a84e-e1535054b574 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:22.538606 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 683/1361] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:35:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:35:22.543547 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ab282-3953-42b0-b643-a1384c38dfa1 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:22.543955 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ab282-3953-42b0-b643-a1384c38dfa1 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:22.544093 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ab282-3953-42b0-b643-a1384c38dfa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:22.614065 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-ec1ab282-3953-42b0-b643-a1384c38dfa1 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:35:22.615297 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-ec1ab282-3953-42b0-b643-a1384c38dfa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:22.655484 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:35:22.725458 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 679/1362] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:35:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:35:23.020316 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3f0371-ba34-4aae-a84e-e1535054b574 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:35:23.021974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-eb3f0371-ba34-4aae-a84e-e1535054b574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:23.102500 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:35:23.189840 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3f0371-ba34-4aae-a84e-e1535054b574 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:35:23.313229 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e8b8d-d19c-417e-bc97-751026aa1b2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:35:23.313838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e8b8d-d19c-417e-bc97-751026aa1b2f service nova] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:23.314151 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e8b8d-d19c-417e-bc97-751026aa1b2f service nova] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:23.314375 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e8b8d-d19c-417e-bc97-751026aa1b2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:23.316055 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-e20e8b8d-d19c-417e-bc97-751026aa1b2f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:23.341185 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:35:23.430576 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20e8b8d-d19c-417e-bc97-751026aa1b2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:35:23.433106 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20e8b8d-d19c-417e-bc97-751026aa1b2f service nova] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:35:23.455907 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 684/1363] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:35:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:35:25.755201 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723b4f4-5c58-407a-b384-b17528d08109 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:25.755743 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723b4f4-5c58-407a-b384-b17528d08109 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:25.755743 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723b4f4-5c58-407a-b384-b17528d08109 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:25.757107 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 680/1364] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:35:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:35:25.762602 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580df367-5e5c-4e5a-a982-3214ee1cce52 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:25.763058 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580df367-5e5c-4e5a-a982-3214ee1cce52 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:25.763141 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580df367-5e5c-4e5a-a982-3214ee1cce52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:25.834562 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-580df367-5e5c-4e5a-a982-3214ee1cce52 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:35:25.835875 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-580df367-5e5c-4e5a-a982-3214ee1cce52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:25.937735 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:35:25.940903 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 685/1365] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:35:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:35:26.236615 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e40533-d966-43df-891d-45343ae681df None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:26.237169 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e40533-d966-43df-891d-45343ae681df None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:26.237629 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e40533-d966-43df-891d-45343ae681df None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:26.239589 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 681/1366] 10.222.0.30 () {54 vars in 962 bytes} [Tue Aug 30 14:35:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:35:26.245733 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf619ed-7317-4b5d-af5f-42edb5b33c2e None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:26.246288 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf619ed-7317-4b5d-af5f-42edb5b33c2e None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:26.246636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf619ed-7317-4b5d-af5f-42edb5b33c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:26.334111 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-3bf619ed-7317-4b5d-af5f-42edb5b33c2e None None] MFA Rules not processed for user `8878de66a03044e38e507028853a0cd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:35:26.335366 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-3bf619ed-7317-4b5d-af5f-42edb5b33c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:26.460338 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 686/1367] 10.222.0.30 () {58 vars in 1059 bytes} [Tue Aug 30 14:35:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:35:26.469015 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e40533-d966-43df-891d-45343ae681df None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:35:26.470506 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-23e40533-d966-43df-891d-45343ae681df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:26.627035 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e40533-d966-43df-891d-45343ae681df None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:35:26.648458 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:35:26.746555 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e03ebbb-d92e-4a59-8228-e61110a91008 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:35:26.747163 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03ebbb-d92e-4a59-8228-e61110a91008 service nova] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:26.747478 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03ebbb-d92e-4a59-8228-e61110a91008 service nova] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:26.747748 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03ebbb-d92e-4a59-8228-e61110a91008 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:26.749868 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-0e03ebbb-d92e-4a59-8228-e61110a91008 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:26.848124 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e03ebbb-d92e-4a59-8228-e61110a91008 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:35:26.849724 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e03ebbb-d92e-4a59-8228-e61110a91008 service nova] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:35:26.867990 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 682/1368] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:35:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:35:27.393076 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1da22-87ee-4293-978f-94055cbb4d55 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:27.393637 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1da22-87ee-4293-978f-94055cbb4d55 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:27.393637 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1da22-87ee-4293-978f-94055cbb4d55 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:27.395104 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 687/1369] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:35:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:35:27.400256 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd78cb04-c9b8-46ea-b2a1-6be6965e9f57 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:27.400540 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd78cb04-c9b8-46ea-b2a1-6be6965e9f57 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:27.400837 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd78cb04-c9b8-46ea-b2a1-6be6965e9f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:27.434968 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:35:27.473732 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-dd78cb04-c9b8-46ea-b2a1-6be6965e9f57 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:35:27.474875 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-dd78cb04-c9b8-46ea-b2a1-6be6965e9f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:27.583096 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 683/1370] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:35:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:35:28.135648 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ce0de-5f54-4d46-8344-741fec87fd08 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:28.135931 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ce0de-5f54-4d46-8344-741fec87fd08 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:28.136211 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ce0de-5f54-4d46-8344-741fec87fd08 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:28.137731 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 688/1371] 10.222.0.30 () {54 vars in 964 bytes} [Tue Aug 30 14:35:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:35:28.142952 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92eabd8-1e58-4248-a053-3085d2f4554b None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:28.143420 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92eabd8-1e58-4248-a053-3085d2f4554b None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:28.143597 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92eabd8-1e58-4248-a053-3085d2f4554b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:28.230957 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-d92eabd8-1e58-4248-a053-3085d2f4554b None None] MFA Rules not processed for user `6fda7075198d48da86aa9c0dab6ff7ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:35:28.232217 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-d92eabd8-1e58-4248-a053-3085d2f4554b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:28.285730 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:35:28.320480 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 684/1372] 10.222.0.30 () {58 vars in 1061 bytes} [Tue Aug 30 14:35:28 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:35:28.328831 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69ce0de-5f54-4d46-8344-741fec87fd08 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:35:28.330716 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-f69ce0de-5f54-4d46-8344-741fec87fd08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:28.384695 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:35:28.443524 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69ce0de-5f54-4d46-8344-741fec87fd08 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:35:28.529102 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbe759c-fb9b-48c0-b819-b4d3fbff55cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:35:28.529807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe759c-fb9b-48c0-b819-b4d3fbff55cd service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:28.530085 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe759c-fb9b-48c0-b819-b4d3fbff55cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:28.530391 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe759c-fb9b-48c0-b819-b4d3fbff55cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:28.532217 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-abbe759c-fb9b-48c0-b819-b4d3fbff55cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:28.640182 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:35:28.654873 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbe759c-fb9b-48c0-b819-b4d3fbff55cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:35:28.656489 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbe759c-fb9b-48c0-b819-b4d3fbff55cd service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:35:28.678421 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 689/1373] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:35:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:35:28.852606 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d7d76-b69e-4724-b882-e67b3eb1d1c4 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:28.853296 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d7d76-b69e-4724-b882-e67b3eb1d1c4 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:28.853650 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d7d76-b69e-4724-b882-e67b3eb1d1c4 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:28.855821 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 685/1374] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:35:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:35:28.861666 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac78f4-8073-4173-a308-623595e99e16 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:28.861955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac78f4-8073-4173-a308-623595e99e16 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:28.862498 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac78f4-8073-4173-a308-623595e99e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:28.942650 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-79ac78f4-8073-4173-a308-623595e99e16 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:35:28.943812 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-79ac78f4-8073-4173-a308-623595e99e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:29.053024 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 690/1375] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:35:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:35:29.343903 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129d7d76-b69e-4724-b882-e67b3eb1d1c4 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:35:29.345499 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-129d7d76-b69e-4724-b882-e67b3eb1d1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:29.505201 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129d7d76-b69e-4724-b882-e67b3eb1d1c4 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:35:29.556120 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:35:29.617934 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459bb1b8-6c5d-4cc7-bbe5-a77a710c8efc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:35:29.618545 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459bb1b8-6c5d-4cc7-bbe5-a77a710c8efc service nova] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:29.618820 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459bb1b8-6c5d-4cc7-bbe5-a77a710c8efc service nova] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:29.619091 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459bb1b8-6c5d-4cc7-bbe5-a77a710c8efc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:29.620714 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-459bb1b8-6c5d-4cc7-bbe5-a77a710c8efc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:29.712922 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459bb1b8-6c5d-4cc7-bbe5-a77a710c8efc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:35:29.714626 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459bb1b8-6c5d-4cc7-bbe5-a77a710c8efc service nova] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:35:29.733075 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 686/1376] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:35:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:35:31.673611 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1311d-65bc-418f-8bf7-dd07a23107aa None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:31.674090 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1311d-65bc-418f-8bf7-dd07a23107aa None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:31.674180 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1311d-65bc-418f-8bf7-dd07a23107aa None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:31.675539 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 691/1377] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:35:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:35:31.680818 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41126f03-f0f9-4643-89c4-68ca4077abc3 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:31.681205 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41126f03-f0f9-4643-89c4-68ca4077abc3 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:31.681365 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41126f03-f0f9-4643-89c4-68ca4077abc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:31.754557 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-41126f03-f0f9-4643-89c4-68ca4077abc3 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:35:31.754557 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-41126f03-f0f9-4643-89c4-68ca4077abc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:31.766188 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:35:31.861872 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 687/1378] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:35:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:35:32.159192 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd1311d-65bc-418f-8bf7-dd07a23107aa None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:35:32.160703 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-2bd1311d-65bc-418f-8bf7-dd07a23107aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:32.254426 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:35:32.353611 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd1311d-65bc-418f-8bf7-dd07a23107aa None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:35:32.473813 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c260a979-d487-46f8-aa51-b9ee6b7bb00e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:35:32.474339 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c260a979-d487-46f8-aa51-b9ee6b7bb00e service nova] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:32.474614 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c260a979-d487-46f8-aa51-b9ee6b7bb00e service nova] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:32.474876 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c260a979-d487-46f8-aa51-b9ee6b7bb00e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:32.476485 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-c260a979-d487-46f8-aa51-b9ee6b7bb00e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:32.498287 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:35:32.582012 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c260a979-d487-46f8-aa51-b9ee6b7bb00e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:35:32.583896 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c260a979-d487-46f8-aa51-b9ee6b7bb00e service nova] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:35:32.607363 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 692/1379] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:35:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:35:34.576394 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c0e90-71b0-4b44-a476-6abd54f73f3b None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:34.576840 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c0e90-71b0-4b44-a476-6abd54f73f3b None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:34.577862 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c0e90-71b0-4b44-a476-6abd54f73f3b None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:34.580244 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 688/1380] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:35:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:35:34.584374 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035fc84e-2df7-4d0d-bc29-45e5bda0544b None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:34.584791 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035fc84e-2df7-4d0d-bc29-45e5bda0544b None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:34.584900 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035fc84e-2df7-4d0d-bc29-45e5bda0544b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:34.659469 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-035fc84e-2df7-4d0d-bc29-45e5bda0544b None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:35:34.660821 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-035fc84e-2df7-4d0d-bc29-45e5bda0544b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:34.762384 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:35:34.770188 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 693/1381] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:35:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:35:35.068506 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c0e90-71b0-4b44-a476-6abd54f73f3b None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:35:35.070113 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-153c0e90-71b0-4b44-a476-6abd54f73f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:35.224185 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:35:35.236266 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c0e90-71b0-4b44-a476-6abd54f73f3b None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:35:35.364276 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d6a89-b89c-49f5-a2e4-4cb384a122c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:35:35.364946 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d6a89-b89c-49f5-a2e4-4cb384a122c6 service nova] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:35.365255 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d6a89-b89c-49f5-a2e4-4cb384a122c6 service nova] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:35.365518 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d6a89-b89c-49f5-a2e4-4cb384a122c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:35.367097 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-b54d6a89-b89c-49f5-a2e4-4cb384a122c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:35.470986 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54d6a89-b89c-49f5-a2e4-4cb384a122c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:35:35.472611 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54d6a89-b89c-49f5-a2e4-4cb384a122c6 service nova] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:35:35.483955 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:35:35.494135 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 689/1382] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:35:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:35:37.442584 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc9696-4396-4d87-83a1-56634662dbfa None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:37.443325 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc9696-4396-4d87-83a1-56634662dbfa None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:37.443325 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc9696-4396-4d87-83a1-56634662dbfa None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:37.444447 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 694/1383] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:35:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:35:37.451326 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e18dbd1-68ec-4da8-876c-b94982fd794d None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:37.451780 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e18dbd1-68ec-4da8-876c-b94982fd794d None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:37.452524 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e18dbd1-68ec-4da8-876c-b94982fd794d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:37.525763 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-3e18dbd1-68ec-4da8-876c-b94982fd794d None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:35:37.527066 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-3e18dbd1-68ec-4da8-876c-b94982fd794d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:37.634275 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 690/1384] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:35:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:35:37.925081 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc9696-4396-4d87-83a1-56634662dbfa None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:35:37.926685 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-6dfc9696-4396-4d87-83a1-56634662dbfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:38.079097 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc9696-4396-4d87-83a1-56634662dbfa None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:35:38.153557 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:35:38.204651 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708e2e55-7516-4877-ba14-5b1476c66547 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:35:38.205203 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e2e55-7516-4877-ba14-5b1476c66547 service nova] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:38.205522 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e2e55-7516-4877-ba14-5b1476c66547 service nova] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:38.209048 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e2e55-7516-4877-ba14-5b1476c66547 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:38.209048 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-708e2e55-7516-4877-ba14-5b1476c66547 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:38.301577 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708e2e55-7516-4877-ba14-5b1476c66547 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:35:38.303169 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708e2e55-7516-4877-ba14-5b1476c66547 service nova] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:35:38.322549 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 695/1385] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:35:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:35:41.151453 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a619b-5b8d-4bf3-98b4-37f7dac8a9cb None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:41.152016 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a619b-5b8d-4bf3-98b4-37f7dac8a9cb None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:41.152172 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a619b-5b8d-4bf3-98b4-37f7dac8a9cb None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:41.153942 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 691/1386] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:35:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:35:41.163849 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd3a5b-4a17-4efb-b0c7-239ef62f61d0 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:41.164400 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd3a5b-4a17-4efb-b0c7-239ef62f61d0 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:41.164544 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd3a5b-4a17-4efb-b0c7-239ef62f61d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:41.258503 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-91bd3a5b-4a17-4efb-b0c7-239ef62f61d0 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:35:41.259766 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-91bd3a5b-4a17-4efb-b0c7-239ef62f61d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:41.294941 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:35:41.374229 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 696/1387] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:35:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:35:41.793905 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2a619b-5b8d-4bf3-98b4-37f7dac8a9cb None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:35:41.795573 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-db2a619b-5b8d-4bf3-98b4-37f7dac8a9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:41.881868 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:35:41.954035 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2a619b-5b8d-4bf3-98b4-37f7dac8a9cb None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:35:42.071833 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f3b76-9a0b-4f0d-8613-36d6f1b2418c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:35:42.072448 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f3b76-9a0b-4f0d-8613-36d6f1b2418c service nova] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:42.072729 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f3b76-9a0b-4f0d-8613-36d6f1b2418c service nova] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:42.072991 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f3b76-9a0b-4f0d-8613-36d6f1b2418c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:42.074589 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-e38f3b76-9a0b-4f0d-8613-36d6f1b2418c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:42.113416 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:35:42.168248 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38f3b76-9a0b-4f0d-8613-36d6f1b2418c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:35:42.169844 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38f3b76-9a0b-4f0d-8613-36d6f1b2418c service nova] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:35:42.189285 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 692/1388] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:35:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:35:43.821906 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55311fff-ce0e-4a7d-aa79-cbff812a9f4c None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:43.822428 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55311fff-ce0e-4a7d-aa79-cbff812a9f4c None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:43.822540 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55311fff-ce0e-4a7d-aa79-cbff812a9f4c None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:43.823873 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 697/1389] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:35:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:35:43.829763 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f750ce-435d-4388-8420-e7a2f89db46c None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:43.830177 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f750ce-435d-4388-8420-e7a2f89db46c None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:43.830444 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f750ce-435d-4388-8420-e7a2f89db46c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:43.897488 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-09f750ce-435d-4388-8420-e7a2f89db46c None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:35:43.898708 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-09f750ce-435d-4388-8420-e7a2f89db46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:43.997516 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 693/1390] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:35:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:35:44.128617 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff0bf2-dd59-4739-977b-9e74e88c118d None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:44.128905 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff0bf2-dd59-4739-977b-9e74e88c118d None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:44.129178 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff0bf2-dd59-4739-977b-9e74e88c118d None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:44.130564 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 698/1391] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:35:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:35:44.135923 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd481f-559a-49d9-a8a2-d4d8f7d0d273 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:44.136205 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd481f-559a-49d9-a8a2-d4d8f7d0d273 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:44.136476 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd481f-559a-49d9-a8a2-d4d8f7d0d273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:44.154238 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:35:44.206242 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-10bd481f-559a-49d9-a8a2-d4d8f7d0d273 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:35:44.207488 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-10bd481f-559a-49d9-a8a2-d4d8f7d0d273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:44.306755 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24630f62-5f7d-466a-bb1c-ce58cec5d0c0 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:44.306925 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 694/1392] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:35:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:35:44.307053 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24630f62-5f7d-466a-bb1c-ce58cec5d0c0 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:44.307345 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24630f62-5f7d-466a-bb1c-ce58cec5d0c0 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:44.308969 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 699/1393] 10.222.0.30 () {54 vars in 967 bytes} [Tue Aug 30 14:35:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:35:44.314605 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754792e0-f519-49ce-b4fb-f683b770e210 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:44.314894 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754792e0-f519-49ce-b4fb-f683b770e210 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:44.315168 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754792e0-f519-49ce-b4fb-f683b770e210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:44.391485 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-754792e0-f519-49ce-b4fb-f683b770e210 None None] MFA Rules not processed for user `6889402e54424bbd8d42c7bb759cf700`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:35:44.394599 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-754792e0-f519-49ce-b4fb-f683b770e210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:44.429817 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:35:44.474111 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 695/1394] 10.222.0.30 () {58 vars in 1064 bytes} [Tue Aug 30 14:35:44 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:35:44.481776 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24630f62-5f7d-466a-bb1c-ce58cec5d0c0 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:35:44.484041 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-24630f62-5f7d-466a-bb1c-ce58cec5d0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:44.589851 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24630f62-5f7d-466a-bb1c-ce58cec5d0c0 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:35:44.613223 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754792e0-f519-49ce-b4fb-f683b770e210 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:35:44.614465 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-754792e0-f519-49ce-b4fb-f683b770e210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:44.651609 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:35:44.668505 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f617c75-650e-4fc1-89e7-ded662d5e769 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6889402e54424bbd8d42c7bb759cf700', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:35:44.669109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f617c75-650e-4fc1-89e7-ded662d5e769 service glance] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:44.669382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f617c75-650e-4fc1-89e7-ded662d5e769 service glance] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:44.669648 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f617c75-650e-4fc1-89e7-ded662d5e769 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:44.671891 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-0f617c75-650e-4fc1-89e7-ded662d5e769 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:44.736526 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754792e0-f519-49ce-b4fb-f683b770e210 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:35:44.783140 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f617c75-650e-4fc1-89e7-ded662d5e769 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:35:44.784634 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f617c75-650e-4fc1-89e7-ded662d5e769 service glance] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:35:44.799620 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:35:44.803826 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 700/1395] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:35:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:35:44.850666 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202f64ec-93a2-4b05-8f72-33de5bbe87b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:35:44.851304 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f64ec-93a2-4b05-8f72-33de5bbe87b3 service nova] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:44.851586 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f64ec-93a2-4b05-8f72-33de5bbe87b3 service nova] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:44.851851 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f64ec-93a2-4b05-8f72-33de5bbe87b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:44.853432 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-202f64ec-93a2-4b05-8f72-33de5bbe87b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:44.944895 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202f64ec-93a2-4b05-8f72-33de5bbe87b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:35:44.946486 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202f64ec-93a2-4b05-8f72-33de5bbe87b3 service nova] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:35:44.964790 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 696/1396] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:35:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:35:45.548742 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74edca5-78f9-470e-a370-daa8206682a8 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:45.549243 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74edca5-78f9-470e-a370-daa8206682a8 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:45.549349 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74edca5-78f9-470e-a370-daa8206682a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:45.616549 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-d74edca5-78f9-470e-a370-daa8206682a8 None None] MFA Rules not processed for user `f60bee471e954932a928ab2cf7d66c82`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:35:45.617763 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-d74edca5-78f9-470e-a370-daa8206682a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:45.660495 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:35:45.706734 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 701/1397] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Aug 30 14:35:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:35:45.752127 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425ccab-b73f-485b-a623-2fb9ec7960c3 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:45.752408 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425ccab-b73f-485b-a623-2fb9ec7960c3 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:45.752676 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425ccab-b73f-485b-a623-2fb9ec7960c3 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:45.754038 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 697/1398] 10.222.0.30 () {54 vars in 943 bytes} [Tue Aug 30 14:35:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:35:45.760326 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a035aac-1376-4103-a35e-5ee8d11315a7 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:45.760614 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a035aac-1376-4103-a35e-5ee8d11315a7 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:45.760890 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a035aac-1376-4103-a35e-5ee8d11315a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:45.837803 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-3a035aac-1376-4103-a35e-5ee8d11315a7 None None] MFA Rules not processed for user `a18b13e3708144e1b5e8df80ec3a4e83`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:35:45.839167 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-3a035aac-1376-4103-a35e-5ee8d11315a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:45.945581 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 702/1399] 10.222.0.30 () {58 vars in 1040 bytes} [Tue Aug 30 14:35:45 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:35:45.954876 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8425ccab-b73f-485b-a623-2fb9ec7960c3 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:35:45.956416 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-8425ccab-b73f-485b-a623-2fb9ec7960c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:46.034597 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:35:46.096558 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8425ccab-b73f-485b-a623-2fb9ec7960c3 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:35:46.202646 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb634d8-af6e-4c1f-9cfc-a182904ed1fe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a18b13e3708144e1b5e8df80ec3a4e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:35:46.203394 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb634d8-af6e-4c1f-9cfc-a182904ed1fe service swift] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:46.203675 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb634d8-af6e-4c1f-9cfc-a182904ed1fe service swift] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:46.203942 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb634d8-af6e-4c1f-9cfc-a182904ed1fe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:46.205599 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-8fb634d8-af6e-4c1f-9cfc-a182904ed1fe service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:46.270673 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:35:46.294114 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb634d8-af6e-4c1f-9cfc-a182904ed1fe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:35:46.295936 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb634d8-af6e-4c1f-9cfc-a182904ed1fe service swift] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:35:46.304030 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 698/1400] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Aug 30 14:35:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 14:35:46.947689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8134305-f2ba-4250-ae1d-60e7dcc7ef5a None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:46.948259 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8134305-f2ba-4250-ae1d-60e7dcc7ef5a None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:46.948259 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8134305-f2ba-4250-ae1d-60e7dcc7ef5a None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:46.952120 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 703/1401] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:35:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:35:46.955366 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e06d4a-82fd-4859-93dc-dc4073e9aabc None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:46.956328 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e06d4a-82fd-4859-93dc-dc4073e9aabc None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:46.956603 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e06d4a-82fd-4859-93dc-dc4073e9aabc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:47.036640 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-49e06d4a-82fd-4859-93dc-dc4073e9aabc None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:35:47.037850 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-49e06d4a-82fd-4859-93dc-dc4073e9aabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:47.152013 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 699/1402] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:35:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:35:47.450354 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8134305-f2ba-4250-ae1d-60e7dcc7ef5a None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:35:47.452358 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-b8134305-f2ba-4250-ae1d-60e7dcc7ef5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:47.467187 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:35:47.620343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8134305-f2ba-4250-ae1d-60e7dcc7ef5a None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:35:47.720180 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:35:47.750169 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136b1471-845e-439f-88a7-e03b5505d0bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:35:47.750808 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b1471-845e-439f-88a7-e03b5505d0bd service nova] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:47.751087 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b1471-845e-439f-88a7-e03b5505d0bd service nova] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:47.751387 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b1471-845e-439f-88a7-e03b5505d0bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:47.752993 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-136b1471-845e-439f-88a7-e03b5505d0bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:47.857521 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136b1471-845e-439f-88a7-e03b5505d0bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:35:47.860042 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136b1471-845e-439f-88a7-e03b5505d0bd service nova] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:35:47.881064 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 704/1403] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:35:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:35:49.848818 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d6fc0-dfa0-473a-a43b-f3c2d291122d None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:49.849412 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d6fc0-dfa0-473a-a43b-f3c2d291122d None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:49.849412 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d6fc0-dfa0-473a-a43b-f3c2d291122d None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:49.854594 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 700/1404] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:35:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:35:49.859693 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7419206-e6f6-47e1-aa7c-3c30b724e7a3 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:49.860216 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7419206-e6f6-47e1-aa7c-3c30b724e7a3 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:49.860317 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7419206-e6f6-47e1-aa7c-3c30b724e7a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:49.933553 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-d7419206-e6f6-47e1-aa7c-3c30b724e7a3 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:35:49.934702 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-d7419206-e6f6-47e1-aa7c-3c30b724e7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:49.993296 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:35:50.047793 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 705/1405] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:35:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:35:50.343833 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d6fc0-dfa0-473a-a43b-f3c2d291122d None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:35:50.345883 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-6f9d6fc0-dfa0-473a-a43b-f3c2d291122d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:50.409485 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:35:50.530166 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d6fc0-dfa0-473a-a43b-f3c2d291122d None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:35:50.669523 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c6099-da02-4305-9c6b-d1d1c26fa97c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:35:50.671243 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c6099-da02-4305-9c6b-d1d1c26fa97c service nova] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:50.671541 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c6099-da02-4305-9c6b-d1d1c26fa97c service nova] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:50.671817 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c6099-da02-4305-9c6b-d1d1c26fa97c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:50.673461 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-282c6099-da02-4305-9c6b-d1d1c26fa97c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:50.687412 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:35:50.777621 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282c6099-da02-4305-9c6b-d1d1c26fa97c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:35:50.779823 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282c6099-da02-4305-9c6b-d1d1c26fa97c service nova] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:35:50.802643 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 701/1406] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:35:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:35:52.823525 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e9a7d2-d8ce-4d79-8be1-48555fbc5173 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:52.824086 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e9a7d2-d8ce-4d79-8be1-48555fbc5173 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:52.824192 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e9a7d2-d8ce-4d79-8be1-48555fbc5173 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:52.829908 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 706/1407] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:35:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:35:52.831535 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4708cc1-eb5f-4d7b-a284-427d38ada473 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:52.831955 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4708cc1-eb5f-4d7b-a284-427d38ada473 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:52.832089 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4708cc1-eb5f-4d7b-a284-427d38ada473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:52.907292 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-e4708cc1-eb5f-4d7b-a284-427d38ada473 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:35:52.908543 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-e4708cc1-eb5f-4d7b-a284-427d38ada473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:53.003111 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:35:53.016137 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 702/1408] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:35:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:35:53.308065 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e9a7d2-d8ce-4d79-8be1-48555fbc5173 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:35:53.310089 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-47e9a7d2-d8ce-4d79-8be1-48555fbc5173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:53.475021 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e9a7d2-d8ce-4d79-8be1-48555fbc5173 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:35:53.513858 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:35:53.605656 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b4181a-26ed-4822-8685-076284e5561c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:35:53.606258 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b4181a-26ed-4822-8685-076284e5561c service nova] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:35:53.606533 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b4181a-26ed-4822-8685-076284e5561c service nova] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:35:53.606807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b4181a-26ed-4822-8685-076284e5561c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:35:53.609004 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-44b4181a-26ed-4822-8685-076284e5561c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:35:53.715445 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b4181a-26ed-4822-8685-076284e5561c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:35:53.717448 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b4181a-26ed-4822-8685-076284e5561c service nova] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:35:53.738792 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 707/1409] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:35:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:36:03.014648 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=4 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:03.736257 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=83 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:08.757344 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b34a2d-dc3b-4799-9ea4-778b8f67359c None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:08.757917 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b34a2d-dc3b-4799-9ea4-778b8f67359c None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:08.757917 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b34a2d-dc3b-4799-9ea4-778b8f67359c None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:08.759326 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 703/1410] 10.222.0.30 () {54 vars in 918 bytes} [Tue Aug 30 14:36:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:36:08.765276 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0864b70-a9de-4322-9e54-06e87542631f None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:08.765697 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0864b70-a9de-4322-9e54-06e87542631f None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:08.765817 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0864b70-a9de-4322-9e54-06e87542631f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:08.840288 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-e0864b70-a9de-4322-9e54-06e87542631f None None] MFA Rules not processed for user `dcf302a170934c25b39699088dac326a`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:36:08.841486 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-e0864b70-a9de-4322-9e54-06e87542631f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:08.947153 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 708/1411] 10.222.0.30 () {58 vars in 1015 bytes} [Tue Aug 30 14:36:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:36:08.961449 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9feef31-776b-4012-a5b3-4c254fc0e7c2 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:08.961809 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9feef31-776b-4012-a5b3-4c254fc0e7c2 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:08.962143 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9feef31-776b-4012-a5b3-4c254fc0e7c2 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:08.963624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 704/1412] 10.222.0.30 () {54 vars in 918 bytes} [Tue Aug 30 14:36:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:36:08.968664 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fbd108-e4b1-47dd-b02a-3f7c2fd49fb6 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:08.968943 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fbd108-e4b1-47dd-b02a-3f7c2fd49fb6 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:08.969212 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fbd108-e4b1-47dd-b02a-3f7c2fd49fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:09.041283 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:09.044914 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-64fbd108-e4b1-47dd-b02a-3f7c2fd49fb6 None None] MFA Rules not processed for user `8878de66a03044e38e507028853a0cd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:36:09.046703 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-64fbd108-e4b1-47dd-b02a-3f7c2fd49fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:09.161298 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 709/1413] 10.222.0.30 () {58 vars in 1015 bytes} [Tue Aug 30 14:36:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:36:11.532065 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40216700-9a61-483e-9cd6-905601045a3b None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:11.532594 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40216700-9a61-483e-9cd6-905601045a3b None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:11.532594 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40216700-9a61-483e-9cd6-905601045a3b None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:11.533935 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 705/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 14:36:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:36:11.539109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eecf6f-5422-4372-9524-8f19f53cd824 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:11.539555 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eecf6f-5422-4372-9524-8f19f53cd824 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:11.539665 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eecf6f-5422-4372-9524-8f19f53cd824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:11.606988 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-a9eecf6f-5422-4372-9524-8f19f53cd824 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:36:11.608451 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-a9eecf6f-5422-4372-9524-8f19f53cd824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:11.663024 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:11.710389 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 710/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 14:36:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:36:11.999541 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40216700-9a61-483e-9cd6-905601045a3b None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:36:12.001098 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-40216700-9a61-483e-9cd6-905601045a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:12.145908 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40216700-9a61-483e-9cd6-905601045a3b None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:36:12.218251 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:12.260233 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60347172-476e-4ab6-9b27-2f2b3894db3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:36:12.260765 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60347172-476e-4ab6-9b27-2f2b3894db3f service nova] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:12.261045 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60347172-476e-4ab6-9b27-2f2b3894db3f service nova] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:12.261318 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60347172-476e-4ab6-9b27-2f2b3894db3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:12.262915 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-60347172-476e-4ab6-9b27-2f2b3894db3f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:12.352401 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60347172-476e-4ab6-9b27-2f2b3894db3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:36:12.354000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60347172-476e-4ab6-9b27-2f2b3894db3f service nova] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:36:12.372210 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 706/1416] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:36:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:36:13.258002 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba1c072-1c3b-4832-bb6f-0d549c4402e8 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:13.258521 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba1c072-1c3b-4832-bb6f-0d549c4402e8 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:13.258618 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba1c072-1c3b-4832-bb6f-0d549c4402e8 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:13.259996 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 711/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 14:36:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:36:13.264969 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3385c-fcfb-4ccd-bf78-65ee3a405543 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:13.265357 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3385c-fcfb-4ccd-bf78-65ee3a405543 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:13.265515 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3385c-fcfb-4ccd-bf78-65ee3a405543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:13.339978 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-9ae3385c-fcfb-4ccd-bf78-65ee3a405543 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:36:13.341186 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-9ae3385c-fcfb-4ccd-bf78-65ee3a405543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:13.368744 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:13.438785 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 707/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 14:36:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:36:13.588161 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba1c072-1c3b-4832-bb6f-0d549c4402e8 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:36:13.589708 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-1ba1c072-1c3b-4832-bb6f-0d549c4402e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:13.736287 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba1c072-1c3b-4832-bb6f-0d549c4402e8 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:36:13.775998 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:13.849966 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e72ae1-63cf-40cf-a236-b03829bdbee0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:36:13.850567 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e72ae1-63cf-40cf-a236-b03829bdbee0 service nova] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:13.850843 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e72ae1-63cf-40cf-a236-b03829bdbee0 service nova] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:13.851112 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e72ae1-63cf-40cf-a236-b03829bdbee0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:13.852737 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-b1e72ae1-63cf-40cf-a236-b03829bdbee0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:13.940533 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e72ae1-63cf-40cf-a236-b03829bdbee0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:36:13.942110 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e72ae1-63cf-40cf-a236-b03829bdbee0 service nova] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:36:13.960360 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 712/1419] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:36:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:36:14.846032 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639e97a-7cef-4332-826f-67067d6f3811 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:14.846632 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639e97a-7cef-4332-826f-67067d6f3811 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:14.846632 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639e97a-7cef-4332-826f-67067d6f3811 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:14.847997 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 708/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 14:36:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:36:14.852842 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c48a25-e750-45a3-89b3-f12fde2f9cb0 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:14.853266 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c48a25-e750-45a3-89b3-f12fde2f9cb0 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:14.853389 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c48a25-e750-45a3-89b3-f12fde2f9cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:14.911146 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:14.933431 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-f9c48a25-e750-45a3-89b3-f12fde2f9cb0 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:36:14.934891 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-f9c48a25-e750-45a3-89b3-f12fde2f9cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:15.030677 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 713/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 14:36:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:36:15.184212 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9639e97a-7cef-4332-826f-67067d6f3811 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:36:15.185772 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-9639e97a-7cef-4332-826f-67067d6f3811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:15.330295 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9639e97a-7cef-4332-826f-67067d6f3811 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:36:15.342696 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:15.444051 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7259a970-0816-4829-802d-d6e2ac6b8712 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:36:15.444567 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7259a970-0816-4829-802d-d6e2ac6b8712 service nova] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:15.444844 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7259a970-0816-4829-802d-d6e2ac6b8712 service nova] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:15.445113 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7259a970-0816-4829-802d-d6e2ac6b8712 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:15.446701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-7259a970-0816-4829-802d-d6e2ac6b8712 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:15.536220 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7259a970-0816-4829-802d-d6e2ac6b8712 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:36:15.537808 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7259a970-0816-4829-802d-d6e2ac6b8712 service nova] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:36:15.556320 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 709/1422] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:36:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:36:16.413422 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89f5d6-4db8-4f9f-a65c-9cc3e3cf6aae None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:16.413987 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89f5d6-4db8-4f9f-a65c-9cc3e3cf6aae None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:16.413987 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89f5d6-4db8-4f9f-a65c-9cc3e3cf6aae None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:16.415348 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 714/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 14:36:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:36:16.421034 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd052c8-9735-436b-94f4-aeb6864a11d3 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:16.421458 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd052c8-9735-436b-94f4-aeb6864a11d3 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:16.421578 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd052c8-9735-436b-94f4-aeb6864a11d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:16.441489 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:16.497362 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-afd052c8-9735-436b-94f4-aeb6864a11d3 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:36:16.498625 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-afd052c8-9735-436b-94f4-aeb6864a11d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:16.596875 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 710/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 14:36:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:36:16.752286 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89f5d6-4db8-4f9f-a65c-9cc3e3cf6aae None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:36:16.753770 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-df89f5d6-4db8-4f9f-a65c-9cc3e3cf6aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:16.863712 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:16.899523 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89f5d6-4db8-4f9f-a65c-9cc3e3cf6aae None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:36:17.010038 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f3c306-43b4-4779-8a62-dbb28290607d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:36:17.010560 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f3c306-43b4-4779-8a62-dbb28290607d service nova] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:17.010835 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f3c306-43b4-4779-8a62-dbb28290607d service nova] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:17.011103 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f3c306-43b4-4779-8a62-dbb28290607d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:17.012705 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-d1f3c306-43b4-4779-8a62-dbb28290607d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:17.086572 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:17.100892 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f3c306-43b4-4779-8a62-dbb28290607d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:36:17.102513 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f3c306-43b4-4779-8a62-dbb28290607d service nova] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:36:17.120950 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 715/1425] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:36:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:36:17.980660 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ae5f2-d85a-46ed-98ac-d87c612865a7 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:17.981205 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ae5f2-d85a-46ed-98ac-d87c612865a7 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:17.981205 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ae5f2-d85a-46ed-98ac-d87c612865a7 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:17.982738 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 711/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 14:36:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:36:17.987566 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c88a3a-d55a-4bfb-ac17-d1a7ac4f9d2d None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:17.987954 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c88a3a-d55a-4bfb-ac17-d1a7ac4f9d2d None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:17.988117 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c88a3a-d55a-4bfb-ac17-d1a7ac4f9d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:18.064746 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-83c88a3a-d55a-4bfb-ac17-d1a7ac4f9d2d None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:36:18.065956 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-83c88a3a-d55a-4bfb-ac17-d1a7ac4f9d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:18.160651 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 716/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 14:36:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:36:18.311605 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8ae5f2-d85a-46ed-98ac-d87c612865a7 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:36:18.313159 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-2e8ae5f2-d85a-46ed-98ac-d87c612865a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:18.396205 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:18.460332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8ae5f2-d85a-46ed-98ac-d87c612865a7 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:36:18.572287 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6908853-bb56-4f97-87bd-d6361598b6db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:36:18.572820 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6908853-bb56-4f97-87bd-d6361598b6db service nova] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:18.573101 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6908853-bb56-4f97-87bd-d6361598b6db service nova] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:18.573375 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6908853-bb56-4f97-87bd-d6361598b6db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:18.574967 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-c6908853-bb56-4f97-87bd-d6361598b6db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:18.632757 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:18.675280 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6908853-bb56-4f97-87bd-d6361598b6db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:36:18.676889 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6908853-bb56-4f97-87bd-d6361598b6db service nova] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:36:18.694901 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 712/1428] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:36:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:36:19.500924 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f8f2e-88b9-401d-bf42-9058981a2ae8 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:19.501764 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f8f2e-88b9-401d-bf42-9058981a2ae8 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:19.501764 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f8f2e-88b9-401d-bf42-9058981a2ae8 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:19.502747 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 717/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 14:36:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:36:19.508029 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583ee5be-3c0c-4a89-84b3-9262e19d4c30 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:19.508468 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583ee5be-3c0c-4a89-84b3-9262e19d4c30 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:19.508576 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583ee5be-3c0c-4a89-84b3-9262e19d4c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:19.583529 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-583ee5be-3c0c-4a89-84b3-9262e19d4c30 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:36:19.584727 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-583ee5be-3c0c-4a89-84b3-9262e19d4c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:19.680154 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 713/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 14:36:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:36:19.833448 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44f8f2e-88b9-401d-bf42-9058981a2ae8 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:36:19.834973 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-d44f8f2e-88b9-401d-bf42-9058981a2ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:19.883980 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:19.981347 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44f8f2e-88b9-401d-bf42-9058981a2ae8 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:36:20.099065 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae757f2-30f0-4725-820c-e9908a145630 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:36:20.099607 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae757f2-30f0-4725-820c-e9908a145630 service nova] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:20.099881 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae757f2-30f0-4725-820c-e9908a145630 service nova] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:20.100146 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae757f2-30f0-4725-820c-e9908a145630 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:20.101741 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-fae757f2-30f0-4725-820c-e9908a145630 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:20.108668 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:20.191480 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae757f2-30f0-4725-820c-e9908a145630 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:36:20.193063 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae757f2-30f0-4725-820c-e9908a145630 service nova] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:36:20.211406 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 718/1431] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:36:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:36:21.012696 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb10234-a84a-4051-88e5-bf13be94d8e8 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:21.013269 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb10234-a84a-4051-88e5-bf13be94d8e8 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:21.013269 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb10234-a84a-4051-88e5-bf13be94d8e8 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:21.014967 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 714/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 14:36:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:36:21.020162 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade5e8a7-5d08-4b09-89a6-6ee5ef518eb5 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:21.020586 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade5e8a7-5d08-4b09-89a6-6ee5ef518eb5 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:21.020730 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade5e8a7-5d08-4b09-89a6-6ee5ef518eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:21.095071 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-ade5e8a7-5d08-4b09-89a6-6ee5ef518eb5 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:36:21.096313 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-ade5e8a7-5d08-4b09-89a6-6ee5ef518eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:21.166328 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:21.196940 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 719/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 14:36:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:36:21.346961 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb10234-a84a-4051-88e5-bf13be94d8e8 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:36:21.348500 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-feb10234-a84a-4051-88e5-bf13be94d8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:21.366593 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:21.493862 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb10234-a84a-4051-88e5-bf13be94d8e8 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:36:21.581469 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:21.608756 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c34a22-4506-41cc-a55b-1144359a6abe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:36:21.609377 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c34a22-4506-41cc-a55b-1144359a6abe service nova] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:21.609678 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c34a22-4506-41cc-a55b-1144359a6abe service nova] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:21.609945 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c34a22-4506-41cc-a55b-1144359a6abe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:21.611841 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-a4c34a22-4506-41cc-a55b-1144359a6abe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:21.702634 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c34a22-4506-41cc-a55b-1144359a6abe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:36:21.704274 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c34a22-4506-41cc-a55b-1144359a6abe service nova] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:36:21.722675 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 715/1434] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:36:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:36:22.524356 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87cac7d-461d-43e4-a901-85c18559d123 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:22.524890 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87cac7d-461d-43e4-a901-85c18559d123 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:22.524890 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87cac7d-461d-43e4-a901-85c18559d123 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:22.526281 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 720/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 14:36:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:36:22.531329 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8734dc-ad2b-4433-8470-d508a46dfdf3 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:22.531754 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8734dc-ad2b-4433-8470-d508a46dfdf3 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:22.531877 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8734dc-ad2b-4433-8470-d508a46dfdf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:22.608984 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-6e8734dc-ad2b-4433-8470-d508a46dfdf3 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:36:22.610199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-6e8734dc-ad2b-4433-8470-d508a46dfdf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:22.651545 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:22.707442 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 716/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 14:36:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:36:22.860271 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87cac7d-461d-43e4-a901-85c18559d123 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:36:22.861793 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-c87cac7d-461d-43e4-a901-85c18559d123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:23.007189 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87cac7d-461d-43e4-a901-85c18559d123 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:36:23.062077 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:23.120499 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f298e0-22b2-4cd8-bbe0-e3e0857ff351 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:36:23.121092 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f298e0-22b2-4cd8-bbe0-e3e0857ff351 service nova] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:23.121366 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f298e0-22b2-4cd8-bbe0-e3e0857ff351 service nova] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:23.121633 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f298e0-22b2-4cd8-bbe0-e3e0857ff351 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:23.124264 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-a6f298e0-22b2-4cd8-bbe0-e3e0857ff351 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:23.215336 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f298e0-22b2-4cd8-bbe0-e3e0857ff351 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:36:23.217468 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f298e0-22b2-4cd8-bbe0-e3e0857ff351 service nova] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:36:23.236549 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 721/1437] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:36:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:36:24.033618 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954f6e8-5ac3-459c-a66c-c8a48644fe65 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:24.034189 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954f6e8-5ac3-459c-a66c-c8a48644fe65 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:24.034189 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954f6e8-5ac3-459c-a66c-c8a48644fe65 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:24.035541 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 717/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 14:36:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:36:24.040576 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f85e8-9f54-4695-8cee-b43558d6a0a3 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:24.041007 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f85e8-9f54-4695-8cee-b43558d6a0a3 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:24.041117 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f85e8-9f54-4695-8cee-b43558d6a0a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:24.117878 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-661f85e8-9f54-4695-8cee-b43558d6a0a3 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:36:24.118999 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-661f85e8-9f54-4695-8cee-b43558d6a0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:24.121003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:24.216208 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 722/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 14:36:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:36:24.368554 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b954f6e8-5ac3-459c-a66c-c8a48644fe65 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:36:24.369998 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-b954f6e8-5ac3-459c-a66c-c8a48644fe65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:24.513570 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b954f6e8-5ac3-459c-a66c-c8a48644fe65 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:36:24.539951 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:24.627487 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b619a50-80f1-40b3-839d-7c82ea4f6a22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:36:24.627992 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b619a50-80f1-40b3-839d-7c82ea4f6a22 service nova] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:24.628269 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b619a50-80f1-40b3-839d-7c82ea4f6a22 service nova] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:24.628535 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b619a50-80f1-40b3-839d-7c82ea4f6a22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:24.630139 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-0b619a50-80f1-40b3-839d-7c82ea4f6a22 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:24.721704 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b619a50-80f1-40b3-839d-7c82ea4f6a22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:36:24.723330 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b619a50-80f1-40b3-839d-7c82ea4f6a22 service nova] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:36:24.741630 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 718/1440] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:36:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:36:26.562582 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5612a8f-49ae-4580-bfb4-bad4689b4335 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:26.563129 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5612a8f-49ae-4580-bfb4-bad4689b4335 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:26.563129 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5612a8f-49ae-4580-bfb4-bad4689b4335 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:26.564523 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 723/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 14:36:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:36:26.569832 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47334d39-aeb5-4d27-b7f7-e35b3da74a07 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:26.570235 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47334d39-aeb5-4d27-b7f7-e35b3da74a07 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:26.570384 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47334d39-aeb5-4d27-b7f7-e35b3da74a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:26.611785 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:26.637363 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-47334d39-aeb5-4d27-b7f7-e35b3da74a07 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:36:26.639149 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-47334d39-aeb5-4d27-b7f7-e35b3da74a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:26.741812 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 719/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 14:36:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:36:27.031382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5612a8f-49ae-4580-bfb4-bad4689b4335 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:36:27.032923 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-f5612a8f-49ae-4580-bfb4-bad4689b4335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:27.137373 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5612a8f-49ae-4580-bfb4-bad4689b4335 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:36:27.169069 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:27.211533 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a3c0ae-1eb8-42ee-a510-23d37957cd0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:36:27.212147 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a3c0ae-1eb8-42ee-a510-23d37957cd0a service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:27.212427 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a3c0ae-1eb8-42ee-a510-23d37957cd0a service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:27.212700 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a3c0ae-1eb8-42ee-a510-23d37957cd0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:27.214442 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-57a3c0ae-1eb8-42ee-a510-23d37957cd0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:27.322294 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a3c0ae-1eb8-42ee-a510-23d37957cd0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:36:27.323834 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a3c0ae-1eb8-42ee-a510-23d37957cd0a service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:36:27.342290 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 724/1443] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:36:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:36:29.418131 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d001033a-6612-4621-a361-6893078568cf None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:29.418657 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d001033a-6612-4621-a361-6893078568cf None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:29.418657 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d001033a-6612-4621-a361-6893078568cf None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:29.420600 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 720/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 14:36:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:36:29.426068 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0391d7e-ea06-4554-b050-dc99fe341430 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:29.426522 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0391d7e-ea06-4554-b050-dc99fe341430 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:29.426631 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0391d7e-ea06-4554-b050-dc99fe341430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:29.492797 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-f0391d7e-ea06-4554-b050-dc99fe341430 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:36:29.494003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-f0391d7e-ea06-4554-b050-dc99fe341430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:29.507154 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:29.589782 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 725/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 14:36:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:36:29.875744 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d001033a-6612-4621-a361-6893078568cf None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:36:29.877288 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-d001033a-6612-4621-a361-6893078568cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:29.980654 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d001033a-6612-4621-a361-6893078568cf None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:36:29.993073 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:30.053815 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f004c32a-606c-466d-9be0-f2f726c0c0b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:36:30.054412 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004c32a-606c-466d-9be0-f2f726c0c0b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:30.054687 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004c32a-606c-466d-9be0-f2f726c0c0b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:30.054952 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004c32a-606c-466d-9be0-f2f726c0c0b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:30.056548 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-f004c32a-606c-466d-9be0-f2f726c0c0b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:30.166125 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f004c32a-606c-466d-9be0-f2f726c0c0b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:36:30.167598 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f004c32a-606c-466d-9be0-f2f726c0c0b5 service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:36:30.185958 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 721/1446] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:36:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:36:32.444345 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21b70f-bf78-443a-b1ca-13ed8eba1e4e None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:32.444901 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21b70f-bf78-443a-b1ca-13ed8eba1e4e None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:32.444901 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c21b70f-bf78-443a-b1ca-13ed8eba1e4e None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:32.446239 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 726/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 14:36:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:36:32.451703 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1299183-1978-4e80-bf44-648e80ad6976 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:32.452149 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1299183-1978-4e80-bf44-648e80ad6976 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:32.452250 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1299183-1978-4e80-bf44-648e80ad6976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:32.501147 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:32.519759 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-b1299183-1978-4e80-bf44-648e80ad6976 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:36:32.520991 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-b1299183-1978-4e80-bf44-648e80ad6976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:32.616308 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 722/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 14:36:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:36:32.900282 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c21b70f-bf78-443a-b1ca-13ed8eba1e4e None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:36:32.901810 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-7c21b70f-bf78-443a-b1ca-13ed8eba1e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:33.004210 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c21b70f-bf78-443a-b1ca-13ed8eba1e4e None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:36:33.044664 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:33.079683 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c770ebdb-8148-49f4-b45f-bc8d92d0f269 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:36:33.080288 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c770ebdb-8148-49f4-b45f-bc8d92d0f269 service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:33.080567 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c770ebdb-8148-49f4-b45f-bc8d92d0f269 service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:33.080832 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c770ebdb-8148-49f4-b45f-bc8d92d0f269 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:33.082420 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-c770ebdb-8148-49f4-b45f-bc8d92d0f269 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:33.193678 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c770ebdb-8148-49f4-b45f-bc8d92d0f269 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:36:33.195045 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c770ebdb-8148-49f4-b45f-bc8d92d0f269 service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:36:33.213679 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 727/1449] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:36:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:36:35.253883 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d897ab-ef15-4498-b7dd-b5dc46fe0355 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:35.254387 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d897ab-ef15-4498-b7dd-b5dc46fe0355 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:35.254504 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d897ab-ef15-4498-b7dd-b5dc46fe0355 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:35.255817 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 723/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 14:36:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:36:35.261140 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd3be1-a049-41fc-b7b4-27a0762176fd None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:35.261563 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd3be1-a049-41fc-b7b4-27a0762176fd None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:35.261713 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd3be1-a049-41fc-b7b4-27a0762176fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:35.328702 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-5cdd3be1-a049-41fc-b7b4-27a0762176fd None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:36:35.329919 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-5cdd3be1-a049-41fc-b7b4-27a0762176fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:35.350759 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:35.434264 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 728/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 14:36:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:36:35.727404 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d897ab-ef15-4498-b7dd-b5dc46fe0355 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:36:35.728977 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-c8d897ab-ef15-4498-b7dd-b5dc46fe0355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:35.833630 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d897ab-ef15-4498-b7dd-b5dc46fe0355 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:36:35.853463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:35.907231 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641a1fc3-6171-4f00-9dae-e0f72a0e13e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:36:35.907851 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a1fc3-6171-4f00-9dae-e0f72a0e13e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:35.908135 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a1fc3-6171-4f00-9dae-e0f72a0e13e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:35.908408 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a1fc3-6171-4f00-9dae-e0f72a0e13e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:35.910001 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-641a1fc3-6171-4f00-9dae-e0f72a0e13e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:36.018474 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641a1fc3-6171-4f00-9dae-e0f72a0e13e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:36:36.019865 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641a1fc3-6171-4f00-9dae-e0f72a0e13e2 service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:36:36.037610 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 724/1452] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:36:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:36:37.996368 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b39a46-9832-4b60-9954-f5b67baf1a1c None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:37.996910 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b39a46-9832-4b60-9954-f5b67baf1a1c None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:37.996910 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b39a46-9832-4b60-9954-f5b67baf1a1c None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:37.998288 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 729/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 14:36:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:36:38.003627 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dcd023-3954-4997-bc0c-ad30117f8ed6 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:38.004005 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dcd023-3954-4997-bc0c-ad30117f8ed6 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:38.004239 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dcd023-3954-4997-bc0c-ad30117f8ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:38.068253 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:38.071125 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-76dcd023-3954-4997-bc0c-ad30117f8ed6 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:36:38.072929 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-76dcd023-3954-4997-bc0c-ad30117f8ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:38.169561 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 725/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 14:36:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:36:38.457377 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b39a46-9832-4b60-9954-f5b67baf1a1c None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:36:38.459350 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-b8b39a46-9832-4b60-9954-f5b67baf1a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:38.562821 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b39a46-9832-4b60-9954-f5b67baf1a1c None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:36:38.611411 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:38.636502 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd5a4b7-f0c3-470b-a1b9-0d302562fbfa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:36:38.637096 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5a4b7-f0c3-470b-a1b9-0d302562fbfa service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:38.637376 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5a4b7-f0c3-470b-a1b9-0d302562fbfa service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:38.637642 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5a4b7-f0c3-470b-a1b9-0d302562fbfa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:38.639910 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-6dd5a4b7-f0c3-470b-a1b9-0d302562fbfa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:38.750930 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd5a4b7-f0c3-470b-a1b9-0d302562fbfa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:36:38.752307 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd5a4b7-f0c3-470b-a1b9-0d302562fbfa service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:36:38.770652 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 730/1455] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:36:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:36:40.731272 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d8da5-d9ea-477c-a4c6-45a4f9daf935 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:40.731811 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d8da5-d9ea-477c-a4c6-45a4f9daf935 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:40.731811 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d8da5-d9ea-477c-a4c6-45a4f9daf935 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:40.733695 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 726/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 14:36:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:36:40.739103 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9cc3bb-3ef8-45ca-8811-43b89a851843 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:40.739559 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9cc3bb-3ef8-45ca-8811-43b89a851843 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:40.739668 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9cc3bb-3ef8-45ca-8811-43b89a851843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:40.806867 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-7d9cc3bb-3ef8-45ca-8811-43b89a851843 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:36:40.808102 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-7d9cc3bb-3ef8-45ca-8811-43b89a851843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:40.837647 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:40.906059 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 731/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 14:36:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:36:41.196737 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474d8da5-d9ea-477c-a4c6-45a4f9daf935 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:36:41.198323 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-474d8da5-d9ea-477c-a4c6-45a4f9daf935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:41.304691 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474d8da5-d9ea-477c-a4c6-45a4f9daf935 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:36:41.333014 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:41.377793 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8752f489-c3ba-4be7-9cca-b0bf3dbf5e6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:36:41.378397 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752f489-c3ba-4be7-9cca-b0bf3dbf5e6f service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:41.378674 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752f489-c3ba-4be7-9cca-b0bf3dbf5e6f service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:41.378946 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752f489-c3ba-4be7-9cca-b0bf3dbf5e6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:41.380558 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-8752f489-c3ba-4be7-9cca-b0bf3dbf5e6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:41.489849 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8752f489-c3ba-4be7-9cca-b0bf3dbf5e6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:36:41.491306 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8752f489-c3ba-4be7-9cca-b0bf3dbf5e6f service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:36:41.509453 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 727/1458] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:36:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:36:43.466023 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce81e7e-9dbf-4797-910c-af09ceb69157 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:43.467051 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce81e7e-9dbf-4797-910c-af09ceb69157 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:43.467051 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce81e7e-9dbf-4797-910c-af09ceb69157 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:43.468416 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 732/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 14:36:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:36:43.473972 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd83a0-29ac-4bcf-81d4-f7bb3f6711a0 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:43.474439 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd83a0-29ac-4bcf-81d4-f7bb3f6711a0 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:43.474621 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd83a0-29ac-4bcf-81d4-f7bb3f6711a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:43.540554 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-60cd83a0-29ac-4bcf-81d4-f7bb3f6711a0 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:36:43.541756 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-60cd83a0-29ac-4bcf-81d4-f7bb3f6711a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:43.549993 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:43.636805 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 728/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 14:36:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:36:43.922231 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce81e7e-9dbf-4797-910c-af09ceb69157 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:36:43.923789 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-dce81e7e-9dbf-4797-910c-af09ceb69157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:44.026234 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce81e7e-9dbf-4797-910c-af09ceb69157 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:36:44.082123 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:44.100881 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1172febc-69ed-4317-aa48-8f56397bff44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:36:44.101478 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1172febc-69ed-4317-aa48-8f56397bff44 service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:44.101756 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1172febc-69ed-4317-aa48-8f56397bff44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:44.102028 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1172febc-69ed-4317-aa48-8f56397bff44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:44.103638 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-1172febc-69ed-4317-aa48-8f56397bff44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:44.212722 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1172febc-69ed-4317-aa48-8f56397bff44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:36:44.214152 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1172febc-69ed-4317-aa48-8f56397bff44 service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:36:44.232990 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 733/1461] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:36:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:36:46.165803 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667801f-5924-4f41-b433-e25d9bbf2c64 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:46.166303 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667801f-5924-4f41-b433-e25d9bbf2c64 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:46.166421 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667801f-5924-4f41-b433-e25d9bbf2c64 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:46.167778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 729/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 14:36:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:36:46.173637 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1acfc1-fd64-4bbc-b755-19d3a2e1b5e1 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:46.174090 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1acfc1-fd64-4bbc-b755-19d3a2e1b5e1 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:46.174188 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1acfc1-fd64-4bbc-b755-19d3a2e1b5e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:46.240668 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-ab1acfc1-fd64-4bbc-b755-19d3a2e1b5e1 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:36:46.241856 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-ab1acfc1-fd64-4bbc-b755-19d3a2e1b5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:46.279609 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:46.339557 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 734/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 14:36:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:36:46.623419 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3667801f-5924-4f41-b433-e25d9bbf2c64 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:36:46.624989 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-3667801f-5924-4f41-b433-e25d9bbf2c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:46.849899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3667801f-5924-4f41-b433-e25d9bbf2c64 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:36:46.886241 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:46.923224 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c3a8e-1aa2-47ba-81e0-f8089abbfaa1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:36:46.923825 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c3a8e-1aa2-47ba-81e0-f8089abbfaa1 service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:46.924097 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c3a8e-1aa2-47ba-81e0-f8089abbfaa1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:46.924360 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c3a8e-1aa2-47ba-81e0-f8089abbfaa1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:46.925916 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-b34c3a8e-1aa2-47ba-81e0-f8089abbfaa1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:47.033127 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34c3a8e-1aa2-47ba-81e0-f8089abbfaa1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:36:47.034537 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34c3a8e-1aa2-47ba-81e0-f8089abbfaa1 service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:36:47.052292 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 730/1464] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:36:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:36:48.974127 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e75d1ff-ecc5-49e2-bf34-d27d224eaf87 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:48.974598 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e75d1ff-ecc5-49e2-bf34-d27d224eaf87 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:48.974688 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e75d1ff-ecc5-49e2-bf34-d27d224eaf87 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:48.976061 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 735/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 14:36:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:36:48.981165 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ace745-4c2f-48f4-a387-10d872c3d3ec None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:48.981608 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ace745-4c2f-48f4-a387-10d872c3d3ec None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:48.981705 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ace745-4c2f-48f4-a387-10d872c3d3ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:49.047817 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-c7ace745-4c2f-48f4-a387-10d872c3d3ec None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:36:49.049024 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-c7ace745-4c2f-48f4-a387-10d872c3d3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:49.067578 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:49.152147 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 731/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 14:36:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:36:49.429287 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e75d1ff-ecc5-49e2-bf34-d27d224eaf87 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:36:49.430812 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-4e75d1ff-ecc5-49e2-bf34-d27d224eaf87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:49.533606 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e75d1ff-ecc5-49e2-bf34-d27d224eaf87 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:36:49.599131 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:49.606131 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77ead9b-3122-4358-a6d7-e288ce71365a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:36:49.606728 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ead9b-3122-4358-a6d7-e288ce71365a service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:49.607013 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ead9b-3122-4358-a6d7-e288ce71365a service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:49.607343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ead9b-3122-4358-a6d7-e288ce71365a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:49.608936 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-b77ead9b-3122-4358-a6d7-e288ce71365a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:49.719121 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77ead9b-3122-4358-a6d7-e288ce71365a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:36:49.720513 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77ead9b-3122-4358-a6d7-e288ce71365a service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:36:49.738252 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 736/1467] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:36:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:36:51.674349 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c6910d-f78b-46b0-979e-fde4c7114806 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:51.674933 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c6910d-f78b-46b0-979e-fde4c7114806 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:51.674933 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c6910d-f78b-46b0-979e-fde4c7114806 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:51.676360 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 732/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 14:36:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:36:51.681478 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132f3b8-2d19-4a09-b84b-e599c46352af None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:51.681870 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132f3b8-2d19-4a09-b84b-e599c46352af None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:51.682035 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132f3b8-2d19-4a09-b84b-e599c46352af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:51.750363 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-d132f3b8-2d19-4a09-b84b-e599c46352af None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:36:51.751589 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-d132f3b8-2d19-4a09-b84b-e599c46352af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:51.800839 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:51.850797 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 737/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 14:36:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:36:52.139039 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c6910d-f78b-46b0-979e-fde4c7114806 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:36:52.140736 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-04c6910d-f78b-46b0-979e-fde4c7114806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:52.246083 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c6910d-f78b-46b0-979e-fde4c7114806 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:36:52.293180 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:36:52.319671 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95beace7-a723-4fd2-93a9-cdcfd57b22b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:36:52.320267 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95beace7-a723-4fd2-93a9-cdcfd57b22b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:36:52.320540 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95beace7-a723-4fd2-93a9-cdcfd57b22b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:36:52.320805 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95beace7-a723-4fd2-93a9-cdcfd57b22b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:36:52.322386 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-95beace7-a723-4fd2-93a9-cdcfd57b22b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:36:52.429224 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95beace7-a723-4fd2-93a9-cdcfd57b22b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:36:52.430566 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95beace7-a723-4fd2-93a9-cdcfd57b22b0 service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:36:52.448993 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 733/1470] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:36:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:37:01.849020 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=22 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:02.446734 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=66 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:12.130090 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e8cc01-fbf3-4a96-ab9c-96a7319a975d None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:12.130704 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e8cc01-fbf3-4a96-ab9c-96a7319a975d None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:12.130913 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e8cc01-fbf3-4a96-ab9c-96a7319a975d None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:12.132338 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 738/1471] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:37:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:12.137426 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2bf64-619e-4d33-850d-3a696757ee50 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:12.137817 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2bf64-619e-4d33-850d-3a696757ee50 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:12.137982 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2bf64-619e-4d33-850d-3a696757ee50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:12.203936 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-c3e2bf64-619e-4d33-850d-3a696757ee50 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:12.205145 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-c3e2bf64-619e-4d33-850d-3a696757ee50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:12.300946 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 734/1472] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:37:12.601406 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e55ef78-de8f-49a5-bb75-d29d08787102 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:12.601695 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e55ef78-de8f-49a5-bb75-d29d08787102 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:12.601969 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e55ef78-de8f-49a5-bb75-d29d08787102 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:12.603348 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 739/1473] 10.222.0.30 () {54 vars in 967 bytes} [Tue Aug 30 14:37:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:12.608271 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27deac-72f3-47be-9180-93bdb69630e0 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:12.608550 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27deac-72f3-47be-9180-93bdb69630e0 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:12.608822 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27deac-72f3-47be-9180-93bdb69630e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:12.683427 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-4d27deac-72f3-47be-9180-93bdb69630e0 None None] MFA Rules not processed for user `6889402e54424bbd8d42c7bb759cf700`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:12.684626 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-4d27deac-72f3-47be-9180-93bdb69630e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:12.706235 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:12.762119 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 735/1474] 10.222.0.30 () {58 vars in 1064 bytes} [Tue Aug 30 14:37:12 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:37:12.770223 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e55ef78-de8f-49a5-bb75-d29d08787102 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:12.771770 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-2e55ef78-de8f-49a5-bb75-d29d08787102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:12.881137 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e55ef78-de8f-49a5-bb75-d29d08787102 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:12.954379 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b369ff-b75c-43e2-a926-7843cae08453 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6889402e54424bbd8d42c7bb759cf700', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:12.954982 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b369ff-b75c-43e2-a926-7843cae08453 service glance] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:12.955297 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b369ff-b75c-43e2-a926-7843cae08453 service glance] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:12.955571 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b369ff-b75c-43e2-a926-7843cae08453 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:12.957168 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-72b369ff-b75c-43e2-a926-7843cae08453 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:13.010389 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:13.066091 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b369ff-b75c-43e2-a926-7843cae08453 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:13.067563 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b369ff-b75c-43e2-a926-7843cae08453 service glance] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:13.086207 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 740/1475] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:37:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:37:15.071863 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1aad38-0d18-489a-acd0-b07c74e31891 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:15.072406 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1aad38-0d18-489a-acd0-b07c74e31891 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:15.072406 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1aad38-0d18-489a-acd0-b07c74e31891 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:15.073785 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 736/1476] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:37:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:37:15.079484 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b61b0-3286-4746-ac12-a1d1bd5fe7aa None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:15.079889 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b61b0-3286-4746-ac12-a1d1bd5fe7aa None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:15.080030 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b61b0-3286-4746-ac12-a1d1bd5fe7aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:15.144959 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-3b8b61b0-3286-4746-ac12-a1d1bd5fe7aa None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:15.146172 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-3b8b61b0-3286-4746-ac12-a1d1bd5fe7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:15.245998 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 741/1477] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:37:15.519093 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1aad38-0d18-489a-acd0-b07c74e31891 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:15.520598 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-3e1aad38-0d18-489a-acd0-b07c74e31891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:15.644926 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1aad38-0d18-489a-acd0-b07c74e31891 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:15.693829 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:15.738302 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed14bdc-d0bb-439c-81a9-fc02e33a9de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:15.738903 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed14bdc-d0bb-439c-81a9-fc02e33a9de4 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:15.739166 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed14bdc-d0bb-439c-81a9-fc02e33a9de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:15.739478 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed14bdc-d0bb-439c-81a9-fc02e33a9de4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:15.740093 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed14bdc-d0bb-439c-81a9-fc02e33a9de4 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:15.741803 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: WARNING py.warnings [None req-aed14bdc-d0bb-439c-81a9-fc02e33a9de4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 14:37:15.741803 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: warnings.warn(msg) Aug 30 14:37:15.741803 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: Aug 30 14:37:15.742933 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed14bdc-d0bb-439c-81a9-fc02e33a9de4 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:15.761927 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 737/1478] 10.222.0.30 () {56 vars in 1108 bytes} [Tue Aug 30 14:37:15 2022] GET /identity/v3/projects => generated 2553 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:37:17.679136 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788b2f1-38b3-402a-bff0-966f8a444ed4 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:17.679692 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788b2f1-38b3-402a-bff0-966f8a444ed4 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:17.679692 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788b2f1-38b3-402a-bff0-966f8a444ed4 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:17.681070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 742/1479] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:37:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:17.687086 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af643b77-7112-4b21-a850-b6f013146f9f None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:17.687552 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af643b77-7112-4b21-a850-b6f013146f9f None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:17.687681 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af643b77-7112-4b21-a850-b6f013146f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:17.752919 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-af643b77-7112-4b21-a850-b6f013146f9f None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:17.754148 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-af643b77-7112-4b21-a850-b6f013146f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:17.850741 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 738/1480] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:37:18.146011 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d788b2f1-38b3-402a-bff0-966f8a444ed4 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:18.147579 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-d788b2f1-38b3-402a-bff0-966f8a444ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:18.157554 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:18.294839 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d788b2f1-38b3-402a-bff0-966f8a444ed4 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:18.374902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:18.408365 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba378091-00ea-4df0-855a-a089161a18cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:18.408903 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba378091-00ea-4df0-855a-a089161a18cb service nova] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:18.409186 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba378091-00ea-4df0-855a-a089161a18cb service nova] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:18.409461 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba378091-00ea-4df0-855a-a089161a18cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:18.411058 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-ba378091-00ea-4df0-855a-a089161a18cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:18.500140 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba378091-00ea-4df0-855a-a089161a18cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:18.501738 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba378091-00ea-4df0-855a-a089161a18cb service nova] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:18.519553 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 743/1481] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:37:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:37:20.610306 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829979dc-1b62-4834-9e1b-243c5a6e514c None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:20.610901 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829979dc-1b62-4834-9e1b-243c5a6e514c None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:20.610901 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829979dc-1b62-4834-9e1b-243c5a6e514c None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:20.612244 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 739/1482] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:37:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:20.617417 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f62b05-8a09-4b5b-bf31-fe2fc2ebebf6 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:20.617874 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f62b05-8a09-4b5b-bf31-fe2fc2ebebf6 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:20.617986 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f62b05-8a09-4b5b-bf31-fe2fc2ebebf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:20.686116 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-20f62b05-8a09-4b5b-bf31-fe2fc2ebebf6 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:20.687371 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-20f62b05-8a09-4b5b-bf31-fe2fc2ebebf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:20.733826 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:20.784851 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 744/1483] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:37:21.071007 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829979dc-1b62-4834-9e1b-243c5a6e514c None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:21.072606 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-829979dc-1b62-4834-9e1b-243c5a6e514c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:21.096999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:21.181201 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829979dc-1b62-4834-9e1b-243c5a6e514c None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:21.256570 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d516cf3a-e280-4516-9546-992c298e13f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:21.257177 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d516cf3a-e280-4516-9546-992c298e13f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:21.257459 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d516cf3a-e280-4516-9546-992c298e13f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:21.257732 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d516cf3a-e280-4516-9546-992c298e13f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:21.259364 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-d516cf3a-e280-4516-9546-992c298e13f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:21.333425 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:21.368649 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d516cf3a-e280-4516-9546-992c298e13f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:21.370084 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d516cf3a-e280-4516-9546-992c298e13f5 service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:21.387839 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 740/1484] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:37:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:37:23.352550 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed5457-3c04-4d14-a7c0-e6ea0d9d72af None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:23.353097 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed5457-3c04-4d14-a7c0-e6ea0d9d72af None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:23.353097 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed5457-3c04-4d14-a7c0-e6ea0d9d72af None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:23.354468 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 745/1485] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:37:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:23.360496 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e794c98-e64b-4f8f-adc1-ecf0c0679031 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:23.360937 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e794c98-e64b-4f8f-adc1-ecf0c0679031 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:23.361052 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e794c98-e64b-4f8f-adc1-ecf0c0679031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:23.427604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-5e794c98-e64b-4f8f-adc1-ecf0c0679031 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:23.428737 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-5e794c98-e64b-4f8f-adc1-ecf0c0679031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:23.524863 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 741/1486] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:37:23.811808 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed5457-3c04-4d14-a7c0-e6ea0d9d72af None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:23.813349 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-13ed5457-3c04-4d14-a7c0-e6ea0d9d72af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:23.916667 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed5457-3c04-4d14-a7c0-e6ea0d9d72af None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:23.986955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:23.988784 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c202734a-bb8b-43bf-ade8-d1eed917b58f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:23.989397 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c202734a-bb8b-43bf-ade8-d1eed917b58f service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:23.989865 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c202734a-bb8b-43bf-ade8-d1eed917b58f service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:23.990197 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c202734a-bb8b-43bf-ade8-d1eed917b58f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:23.992197 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-c202734a-bb8b-43bf-ade8-d1eed917b58f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:24.100202 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c202734a-bb8b-43bf-ade8-d1eed917b58f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:24.101552 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c202734a-bb8b-43bf-ade8-d1eed917b58f service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:24.119912 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 746/1487] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:37:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:37:26.104998 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b3d92-2ad2-49ee-a748-bd4df94ab329 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:26.105570 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b3d92-2ad2-49ee-a748-bd4df94ab329 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:26.105570 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b3d92-2ad2-49ee-a748-bd4df94ab329 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:26.106967 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 742/1488] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:37:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:26.112265 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36822b56-0508-4fcf-9455-587c4ce4daf6 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:26.112714 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36822b56-0508-4fcf-9455-587c4ce4daf6 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:26.112870 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36822b56-0508-4fcf-9455-587c4ce4daf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:26.178489 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-36822b56-0508-4fcf-9455-587c4ce4daf6 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:26.179716 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-36822b56-0508-4fcf-9455-587c4ce4daf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:26.233173 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:26.278525 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 747/1489] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:37:26.568541 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b3d92-2ad2-49ee-a748-bd4df94ab329 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:26.570074 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-145b3d92-2ad2-49ee-a748-bd4df94ab329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:26.606774 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:26.675516 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b3d92-2ad2-49ee-a748-bd4df94ab329 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:26.749181 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5501baf-7411-4fab-8852-c4f72fccb45e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:26.749779 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5501baf-7411-4fab-8852-c4f72fccb45e service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:26.750053 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5501baf-7411-4fab-8852-c4f72fccb45e service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:26.750318 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5501baf-7411-4fab-8852-c4f72fccb45e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:26.751926 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-b5501baf-7411-4fab-8852-c4f72fccb45e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:26.829974 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:26.858557 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5501baf-7411-4fab-8852-c4f72fccb45e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:26.860014 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5501baf-7411-4fab-8852-c4f72fccb45e service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:26.878543 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 743/1490] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:37:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:37:28.980426 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1774b2-e320-48ec-94b2-396c8e78fcb4 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:28.980967 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1774b2-e320-48ec-94b2-396c8e78fcb4 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:28.980967 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1774b2-e320-48ec-94b2-396c8e78fcb4 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:28.982346 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 748/1491] 10.222.0.30 () {54 vars in 927 bytes} [Tue Aug 30 14:37:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:37:28.987674 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c9148-dc7f-4786-a04c-ae9db29be538 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:28.988077 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c9148-dc7f-4786-a04c-ae9db29be538 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:28.988219 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c9148-dc7f-4786-a04c-ae9db29be538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:29.053143 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-c45c9148-dc7f-4786-a04c-ae9db29be538 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:37:29.054341 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-c45c9148-dc7f-4786-a04c-ae9db29be538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:29.155265 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 744/1492] 10.222.0.30 () {58 vars in 1024 bytes} [Tue Aug 30 14:37:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:37:29.449019 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1774b2-e320-48ec-94b2-396c8e78fcb4 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:37:29.450583 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-ed1774b2-e320-48ec-94b2-396c8e78fcb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:29.552954 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1774b2-e320-48ec-94b2-396c8e78fcb4 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:37:29.626279 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f38aca-ba72-44b9-938b-c8321036bdea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:37:29.626876 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f38aca-ba72-44b9-938b-c8321036bdea service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:37:29.627153 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f38aca-ba72-44b9-938b-c8321036bdea service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:37:29.627523 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f38aca-ba72-44b9-938b-c8321036bdea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:37:29.629120 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-f4f38aca-ba72-44b9-938b-c8321036bdea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:37:29.638085 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:29.740894 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f38aca-ba72-44b9-938b-c8321036bdea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:37:29.742253 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f38aca-ba72-44b9-938b-c8321036bdea service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:37:29.760844 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 749/1493] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:37:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:37:39.153285 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=82 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:37:39.758584 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=50 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:49.136954 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d50148a-e326-41a0-88b8-4f7e6c949c05 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:49.137554 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d50148a-e326-41a0-88b8-4f7e6c949c05 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:49.137554 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d50148a-e326-41a0-88b8-4f7e6c949c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:49.211951 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-9d50148a-e326-41a0-88b8-4f7e6c949c05 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:49.213165 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-9d50148a-e326-41a0-88b8-4f7e6c949c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:49.312506 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 745/1494] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:38:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:38:49.320702 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c5ab1-1976-45a8-8ae3-160429fdd850 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:49.321202 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c5ab1-1976-45a8-8ae3-160429fdd850 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:49.321386 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c5ab1-1976-45a8-8ae3-160429fdd850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:49.397440 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-6a3c5ab1-1976-45a8-8ae3-160429fdd850 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:49.398563 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-6a3c5ab1-1976-45a8-8ae3-160429fdd850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:49.495268 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 750/1495] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:38:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:38:49.500599 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d50148a-e326-41a0-88b8-4f7e6c949c05 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:49.501812 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-9d50148a-e326-41a0-88b8-4f7e6c949c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:49.513054 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d50148a-e326-41a0-88b8-4f7e6c949c05 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:49.574277 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:49.608262 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee3cdb-0b5f-4d42-ba94-c13bbe891aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:49.609371 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee3cdb-0b5f-4d42-ba94-c13bbe891aa2 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:49.609648 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee3cdb-0b5f-4d42-ba94-c13bbe891aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:49.609934 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee3cdb-0b5f-4d42-ba94-c13bbe891aa2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:49.610555 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee3cdb-0b5f-4d42-ba94-c13bbe891aa2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:38:49.610855 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee3cdb-0b5f-4d42-ba94-c13bbe891aa2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:49.611896 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: WARNING py.warnings [None req-2bee3cdb-0b5f-4d42-ba94-c13bbe891aa2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 30 14:38:49.611896 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: warnings.warn(msg) Aug 30 14:38:49.611896 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: Aug 30 14:38:49.612791 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee3cdb-0b5f-4d42-ba94-c13bbe891aa2 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:49.623815 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 746/1496] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 14:38:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:38:49.633009 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3c5ab1-1976-45a8-8ae3-160429fdd850 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:49.642549 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3c5ab1-1976-45a8-8ae3-160429fdd850 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:49.704803 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:49.739338 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5281d48-53bf-4d60-9df5-3fec4e2f5ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:49.739947 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5281d48-53bf-4d60-9df5-3fec4e2f5ffe admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:49.740225 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5281d48-53bf-4d60-9df5-3fec4e2f5ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:49.740495 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5281d48-53bf-4d60-9df5-3fec4e2f5ffe admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:49.741136 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5281d48-53bf-4d60-9df5-3fec4e2f5ffe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:49.742920 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: WARNING py.warnings [None req-b5281d48-53bf-4d60-9df5-3fec4e2f5ffe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 14:38:49.742920 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: warnings.warn(msg) Aug 30 14:38:49.742920 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: Aug 30 14:38:49.744176 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5281d48-53bf-4d60-9df5-3fec4e2f5ffe admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:49.801499 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 751/1497] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:38:49 2022] POST /identity/v3/projects => generated 379 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:38:49.805604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee3cdb-0b5f-4d42-ba94-c13bbe891aa2 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:49.807113 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-2bee3cdb-0b5f-4d42-ba94-c13bbe891aa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:49.841227 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee3cdb-0b5f-4d42-ba94-c13bbe891aa2 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:49.935441 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca4e933-cce9-4f16-91be-4ea547534d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:49.936049 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca4e933-cce9-4f16-91be-4ea547534d78 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:49.936350 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca4e933-cce9-4f16-91be-4ea547534d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:49.936666 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca4e933-cce9-4f16-91be-4ea547534d78 admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:49.937505 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca4e933-cce9-4f16-91be-4ea547534d78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:49.938758 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: WARNING py.warnings [None req-3ca4e933-cce9-4f16-91be-4ea547534d78 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 14:38:49.938758 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: warnings.warn(msg) Aug 30 14:38:49.938758 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: Aug 30 14:38:49.939986 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca4e933-cce9-4f16-91be-4ea547534d78 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:49.980934 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 747/1498] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:38:49 2022] POST /identity/v3/users => generated 290 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:38:49.985601 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5281d48-53bf-4d60-9df5-3fec4e2f5ffe admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:49.987118 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-b5281d48-53bf-4d60-9df5-3fec4e2f5ffe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:50.048629 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5281d48-53bf-4d60-9df5-3fec4e2f5ffe admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:50.130124 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:50.141929 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd9c198-3d43-4c49-b8ce-fadf3fee54fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:50.142570 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9c198-3d43-4c49-b8ce-fadf3fee54fa admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:50.142847 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9c198-3d43-4c49-b8ce-fadf3fee54fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:50.143126 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9c198-3d43-4c49-b8ce-fadf3fee54fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:50.143794 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd9c198-3d43-4c49-b8ce-fadf3fee54fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:50.144802 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: WARNING py.warnings [None req-2fd9c198-3d43-4c49-b8ce-fadf3fee54fa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 30 14:38:50.144802 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: warnings.warn(msg) Aug 30 14:38:50.144802 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: Aug 30 14:38:50.145698 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd9c198-3d43-4c49-b8ce-fadf3fee54fa admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:50.158744 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 752/1499] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:38:49 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:38:50.162772 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca4e933-cce9-4f16-91be-4ea547534d78 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:50.164333 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-3ca4e933-cce9-4f16-91be-4ea547534d78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:50.176512 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca4e933-cce9-4f16-91be-4ea547534d78 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:50.201610 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:50.273622 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ba8d2d-4b0e-4fc6-85a0-928ed93a00c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:50.274132 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba8d2d-4b0e-4fc6-85a0-928ed93a00c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:50.274404 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba8d2d-4b0e-4fc6-85a0-928ed93a00c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:50.274669 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba8d2d-4b0e-4fc6-85a0-928ed93a00c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:50.275324 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ba8d2d-4b0e-4fc6-85a0-928ed93a00c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:50.276365 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: WARNING py.warnings [None req-93ba8d2d-4b0e-4fc6-85a0-928ed93a00c1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 30 14:38:50.276365 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: warnings.warn(msg) Aug 30 14:38:50.276365 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: Aug 30 14:38:50.277245 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ba8d2d-4b0e-4fc6-85a0-928ed93a00c1 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:50.290198 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 748/1500] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:38:50 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:38:50.294429 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd9c198-3d43-4c49-b8ce-fadf3fee54fa admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:50.296042 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-2fd9c198-3d43-4c49-b8ce-fadf3fee54fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:50.329524 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd9c198-3d43-4c49-b8ce-fadf3fee54fa admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:50.425056 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219d018f-b47e-4d92-a363-15dc39c2c39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:50.425717 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d018f-b47e-4d92-a363-15dc39c2c39d admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:50.425996 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d018f-b47e-4d92-a363-15dc39c2c39d admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:50.426272 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d018f-b47e-4d92-a363-15dc39c2c39d admin admin] PATH_INFO: `/v3/projects/ed0a9e24cbdf4e1e9bc1d86fe0d7d08f/users/e59a88972c264191a4378240ef0cf326/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:50.448027 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219d018f-b47e-4d92-a363-15dc39c2c39d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed0a9e24cbdf4e1e9bc1d86fe0d7d08f, user_id=e59a88972c264191a4378240ef0cf326, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:50.449045 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: WARNING py.warnings [None req-219d018f-b47e-4d92-a363-15dc39c2c39d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 14:38:50.449045 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: warnings.warn(msg) Aug 30 14:38:50.449045 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: Aug 30 14:38:50.451090 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219d018f-b47e-4d92-a363-15dc39c2c39d admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:50.476433 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 753/1501] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:38:50 2022] PUT /identity/v3/projects/ed0a9e24cbdf4e1e9bc1d86fe0d7d08f/users/e59a88972c264191a4378240ef0cf326/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:38:50.489231 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5841d-6113-4cd6-afd8-677273ba21d4 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:50.489512 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5841d-6113-4cd6-afd8-677273ba21d4 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:50.489780 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5841d-6113-4cd6-afd8-677273ba21d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:50.547149 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-07e5841d-6113-4cd6-afd8-677273ba21d4 None None] MFA Rules not processed for user `e59a88972c264191a4378240ef0cf326`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:50.548320 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-07e5841d-6113-4cd6-afd8-677273ba21d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:50.626963 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 749/1502] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:38:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:38:50.640100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c68e55b-bc79-415a-9bf3-0695319ea067 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:50.640388 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c68e55b-bc79-415a-9bf3-0695319ea067 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:50.640664 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c68e55b-bc79-415a-9bf3-0695319ea067 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:50.642010 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 754/1503] 10.222.0.30 () {54 vars in 964 bytes} [Tue Aug 30 14:38:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:50.646752 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3106d7-0f70-44fe-9ecd-490f661f051f None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:50.647035 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3106d7-0f70-44fe-9ecd-490f661f051f None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:50.647343 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3106d7-0f70-44fe-9ecd-490f661f051f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:50.678282 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:50.722070 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-dd3106d7-0f70-44fe-9ecd-490f661f051f None None] MFA Rules not processed for user `6fda7075198d48da86aa9c0dab6ff7ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:50.723285 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-dd3106d7-0f70-44fe-9ecd-490f661f051f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:50.797534 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 750/1504] 10.222.0.30 () {58 vars in 1061 bytes} [Tue Aug 30 14:38:50 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:38:50.805176 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c68e55b-bc79-415a-9bf3-0695319ea067 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:50.806690 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-0c68e55b-bc79-415a-9bf3-0695319ea067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:50.856943 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:50.910620 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c68e55b-bc79-415a-9bf3-0695319ea067 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:50.984613 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976bf31a-683c-4e70-95c7-3b5146b64291 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:50.985214 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976bf31a-683c-4e70-95c7-3b5146b64291 service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:50.985493 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976bf31a-683c-4e70-95c7-3b5146b64291 service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:50.985767 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976bf31a-683c-4e70-95c7-3b5146b64291 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:50.987416 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-976bf31a-683c-4e70-95c7-3b5146b64291 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:51.085765 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976bf31a-683c-4e70-95c7-3b5146b64291 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:51.087255 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976bf31a-683c-4e70-95c7-3b5146b64291 service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:51.090850 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:51.105508 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 755/1505] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:38:50 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:38:51.150573 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337943b0-f5fd-4317-bdd7-53a356cf3f56 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:51.150860 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337943b0-f5fd-4317-bdd7-53a356cf3f56 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:51.151137 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337943b0-f5fd-4317-bdd7-53a356cf3f56 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:51.152454 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 751/1506] 10.222.0.30 () {52 vars in 830 bytes} [Tue Aug 30 14:38:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:38:51.156277 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976bf31a-683c-4e70-95c7-3b5146b64291 service cinder] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:51.157500 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-976bf31a-683c-4e70-95c7-3b5146b64291 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:51.259844 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976bf31a-683c-4e70-95c7-3b5146b64291 service cinder] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:51.355253 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce23b977-75ee-446c-a731-f2c70e57ebab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:51.355874 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23b977-75ee-446c-a731-f2c70e57ebab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:51.356090 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23b977-75ee-446c-a731-f2c70e57ebab admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:51.356353 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23b977-75ee-446c-a731-f2c70e57ebab admin admin] PATH_INFO: `/v3/users/e59a88972c264191a4378240ef0cf326` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:51.369338 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce23b977-75ee-446c-a731-f2c70e57ebab admin admin] RBAC: Authorizing `identity:delete_user(user_id=e59a88972c264191a4378240ef0cf326)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:51.370361 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: WARNING py.warnings [None req-ce23b977-75ee-446c-a731-f2c70e57ebab admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 14:38:51.370361 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: warnings.warn(msg) Aug 30 14:38:51.370361 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: Aug 30 14:38:51.371643 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce23b977-75ee-446c-a731-f2c70e57ebab admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:51.381436 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:51.437423 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-ce23b977-75ee-446c-a731-f2c70e57ebab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e59a88972c264191a4378240ef0cf326'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:38:51.438229 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-ce23b977-75ee-446c-a731-f2c70e57ebab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e59a88972c264191a4378240ef0cf326'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:38:51.460686 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-ce23b977-75ee-446c-a731-f2c70e57ebab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e59a88972c264191a4378240ef0cf326'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:38:51.474874 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-ce23b977-75ee-446c-a731-f2c70e57ebab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e59a88972c264191a4378240ef0cf326'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:38:51.483965 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-ce23b977-75ee-446c-a731-f2c70e57ebab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e59a88972c264191a4378240ef0cf326'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:38:51.490088 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 756/1507] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:38:51 2022] DELETE /identity/v3/users/e59a88972c264191a4378240ef0cf326 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:38:51.494555 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be861dad-9b8d-4033-8d05-71d4d66bfe4f None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:51.495023 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be861dad-9b8d-4033-8d05-71d4d66bfe4f None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:51.495170 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be861dad-9b8d-4033-8d05-71d4d66bfe4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:51.552197 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-be861dad-9b8d-4033-8d05-71d4d66bfe4f None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:38:51.553384 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-be861dad-9b8d-4033-8d05-71d4d66bfe4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:51.628365 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:51.640975 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 752/1508] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:38:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:38:51.650501 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce23b977-75ee-446c-a731-f2c70e57ebab admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:51.651761 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-ce23b977-75ee-446c-a731-f2c70e57ebab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:51.755013 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce23b977-75ee-446c-a731-f2c70e57ebab admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:51.930605 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:38:51.952776 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9a6115-119c-4dfc-b422-069d82225913 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:51.953377 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a6115-119c-4dfc-b422-069d82225913 service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:51.953654 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a6115-119c-4dfc-b422-069d82225913 service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:51.953926 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a6115-119c-4dfc-b422-069d82225913 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:51.955549 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-4a9a6115-119c-4dfc-b422-069d82225913 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:52.065233 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9a6115-119c-4dfc-b422-069d82225913 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:52.066662 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9a6115-119c-4dfc-b422-069d82225913 service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:52.084611 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 757/1509] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:38:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:38:52.531458 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be861dad-9b8d-4033-8d05-71d4d66bfe4f None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:38:52.532681 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-be861dad-9b8d-4033-8d05-71d4d66bfe4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:38:52.544173 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be861dad-9b8d-4033-8d05-71d4d66bfe4f None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:38:52.645169 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d3ccd1-e5cd-4fb6-9be1-d6b7ce63cc1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:38:52.645795 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d3ccd1-e5cd-4fb6-9be1-d6b7ce63cc1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:38:52.646066 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d3ccd1-e5cd-4fb6-9be1-d6b7ce63cc1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:38:52.646338 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d3ccd1-e5cd-4fb6-9be1-d6b7ce63cc1d admin admin] PATH_INFO: `/v3/projects/ed0a9e24cbdf4e1e9bc1d86fe0d7d08f` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:38:52.656874 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d3ccd1-e5cd-4fb6-9be1-d6b7ce63cc1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed0a9e24cbdf4e1e9bc1d86fe0d7d08f)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:38:52.657887 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: WARNING py.warnings [None req-59d3ccd1-e5cd-4fb6-9be1-d6b7ce63cc1d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 14:38:52.657887 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: warnings.warn(msg) Aug 30 14:38:52.657887 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: Aug 30 14:38:52.659079 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d3ccd1-e5cd-4fb6-9be1-d6b7ce63cc1d admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:38:52.716260 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-59d3ccd1-e5cd-4fb6-9be1-d6b7ce63cc1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed0a9e24cbdf4e1e9bc1d86fe0d7d08f', 'request_id': 'req-59d3ccd1-e5cd-4fb6-9be1-d6b7ce63cc1d'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:38:52.723005 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-59d3ccd1-e5cd-4fb6-9be1-d6b7ce63cc1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed0a9e24cbdf4e1e9bc1d86fe0d7d08f', 'request_id': 'req-59d3ccd1-e5cd-4fb6-9be1-d6b7ce63cc1d'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:38:52.736069 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-59d3ccd1-e5cd-4fb6-9be1-d6b7ce63cc1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed0a9e24cbdf4e1e9bc1d86fe0d7d08f', 'request_id': 'req-59d3ccd1-e5cd-4fb6-9be1-d6b7ce63cc1d'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:38:52.761918 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-59d3ccd1-e5cd-4fb6-9be1-d6b7ce63cc1d admin admin] The token cache is being invalidate because project ed0a9e24cbdf4e1e9bc1d86fe0d7d08f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:38:52.762210 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-59d3ccd1-e5cd-4fb6-9be1-d6b7ce63cc1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:38:52.762736 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-59d3ccd1-e5cd-4fb6-9be1-d6b7ce63cc1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:38:52.764086 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 753/1510] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:38:52 2022] DELETE /identity/v3/projects/ed0a9e24cbdf4e1e9bc1d86fe0d7d08f => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:39:02.082268 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=62 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:39:02.760168 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:40:18.316845 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74687b3-8301-4b14-82f2-cc1d53ab632e None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:18.317457 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74687b3-8301-4b14-82f2-cc1d53ab632e None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:18.317457 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74687b3-8301-4b14-82f2-cc1d53ab632e None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:18.318748 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 758/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 14:40:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:40:18.323644 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d4e95-0bca-486f-9b11-9f27b95e1b0b None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:18.324072 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d4e95-0bca-486f-9b11-9f27b95e1b0b None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:18.324190 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d4e95-0bca-486f-9b11-9f27b95e1b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:18.404134 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-c19d4e95-0bca-486f-9b11-9f27b95e1b0b None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:40:18.405333 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-c19d4e95-0bca-486f-9b11-9f27b95e1b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:40:18.510161 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 754/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 14:40:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:40:18.668943 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74687b3-8301-4b14-82f2-cc1d53ab632e None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:18.671015 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-e74687b3-8301-4b14-82f2-cc1d53ab632e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:40:18.792918 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74687b3-8301-4b14-82f2-cc1d53ab632e None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:18.798646 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f8d2a0-7f81-4c30-81db-5f59b40454e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:18.799266 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8d2a0-7f81-4c30-81db-5f59b40454e6 service nova] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:18.799547 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8d2a0-7f81-4c30-81db-5f59b40454e6 service nova] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:18.799830 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8d2a0-7f81-4c30-81db-5f59b40454e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:18.807785 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f8d2a0-7f81-4c30-81db-5f59b40454e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:40:18.809387 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f8d2a0-7f81-4c30-81db-5f59b40454e6 service nova] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:40:18.819600 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 759/1513] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:40:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:40:19.652002 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dfa6d8-c9d9-4291-96a4-6f787ce49ce8 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:19.652571 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dfa6d8-c9d9-4291-96a4-6f787ce49ce8 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:19.652571 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dfa6d8-c9d9-4291-96a4-6f787ce49ce8 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:19.653925 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 755/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 14:40:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:40:19.659257 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df213a1-2cb6-4a00-8e3e-acf1000f6590 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:19.659705 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df213a1-2cb6-4a00-8e3e-acf1000f6590 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:19.659813 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df213a1-2cb6-4a00-8e3e-acf1000f6590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:19.679493 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-4df213a1-2cb6-4a00-8e3e-acf1000f6590 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:40:19.680701 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-4df213a1-2cb6-4a00-8e3e-acf1000f6590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:40:19.688732 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 760/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 14:40:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:40:19.842451 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dfa6d8-c9d9-4291-96a4-6f787ce49ce8 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:19.857282 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dfa6d8-c9d9-4291-96a4-6f787ce49ce8 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:19.862886 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab80e84-8844-4b42-ac3f-f98da8a34685 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:19.863514 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab80e84-8844-4b42-ac3f-f98da8a34685 service nova] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:19.863789 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab80e84-8844-4b42-ac3f-f98da8a34685 service nova] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:19.864060 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab80e84-8844-4b42-ac3f-f98da8a34685 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:19.872006 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab80e84-8844-4b42-ac3f-f98da8a34685 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:40:19.873574 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab80e84-8844-4b42-ac3f-f98da8a34685 service nova] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:40:19.882924 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 756/1516] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:40:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:40:20.367240 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5197af-273b-4075-af1b-98e77336f430 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:20.367535 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5197af-273b-4075-af1b-98e77336f430 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:20.367813 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5197af-273b-4075-af1b-98e77336f430 None None] PATH_INFO: `/v3` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:20.369109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 761/1517] 10.222.0.58 () {54 vars in 925 bytes} [Tue Aug 30 14:40:20 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:40:20.375830 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b90faa-47a7-4c70-9f36-ab0122cf2f55 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:20.376194 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b90faa-47a7-4c70-9f36-ab0122cf2f55 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:20.376487 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b90faa-47a7-4c70-9f36-ab0122cf2f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:20.436975 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-93b90faa-47a7-4c70-9f36-ab0122cf2f55 None None] MFA Rules not processed for user `dcf302a170934c25b39699088dac326a`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:40:20.438677 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-93b90faa-47a7-4c70-9f36-ab0122cf2f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:40:20.456350 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:40:20.510779 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 757/1518] 10.222.0.58 () {58 vars in 1015 bytes} [Tue Aug 30 14:40:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:40:29.619654 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf44b9-a0b3-4594-a910-20c5c5958999 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:29.620253 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf44b9-a0b3-4594-a910-20c5c5958999 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:29.620253 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf44b9-a0b3-4594-a910-20c5c5958999 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:29.621542 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 762/1519] 10.222.0.30 () {54 vars in 961 bytes} [Tue Aug 30 14:40:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:40:29.626931 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ed2c9-89cb-4ce9-bf58-233ba5299381 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:29.627414 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ed2c9-89cb-4ce9-bf58-233ba5299381 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:29.627521 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ed2c9-89cb-4ce9-bf58-233ba5299381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:29.647374 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-712ed2c9-89cb-4ce9-bf58-233ba5299381 None None] MFA Rules not processed for user `dcf302a170934c25b39699088dac326a`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:40:29.648556 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-712ed2c9-89cb-4ce9-bf58-233ba5299381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:40:29.657887 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 758/1520] 10.222.0.30 () {58 vars in 1058 bytes} [Tue Aug 30 14:40:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:40:29.665420 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf44b9-a0b3-4594-a910-20c5c5958999 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:29.678817 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf44b9-a0b3-4594-a910-20c5c5958999 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:29.683774 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32208099-a440-4726-ab47-95b3c31fc471 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcf302a170934c25b39699088dac326a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:29.684392 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32208099-a440-4726-ab47-95b3c31fc471 service placement] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:29.684674 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32208099-a440-4726-ab47-95b3c31fc471 service placement] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:29.684949 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32208099-a440-4726-ab47-95b3c31fc471 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:29.695477 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32208099-a440-4726-ab47-95b3c31fc471 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:40:29.697251 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32208099-a440-4726-ab47-95b3c31fc471 service placement] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:40:29.709461 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 763/1521] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:40:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:40:31.120485 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7366f1d0-96d9-45ed-ab4d-69587381353e None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:31.121072 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7366f1d0-96d9-45ed-ab4d-69587381353e None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:31.121072 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7366f1d0-96d9-45ed-ab4d-69587381353e None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:31.122371 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 759/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 14:40:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:40:31.127441 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cff19c-bf32-4fdb-af28-d23af30333eb None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:31.127846 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cff19c-bf32-4fdb-af28-d23af30333eb None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:31.127991 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cff19c-bf32-4fdb-af28-d23af30333eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:31.147325 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-32cff19c-bf32-4fdb-af28-d23af30333eb None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:40:31.148517 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-32cff19c-bf32-4fdb-af28-d23af30333eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:40:31.156849 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 764/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 14:40:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:40:31.313491 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7366f1d0-96d9-45ed-ab4d-69587381353e None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:31.329172 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7366f1d0-96d9-45ed-ab4d-69587381353e None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:31.334652 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11508fc0-999a-4a2a-b7b0-013b66eadbf0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:31.335281 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11508fc0-999a-4a2a-b7b0-013b66eadbf0 service nova] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:31.335562 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11508fc0-999a-4a2a-b7b0-013b66eadbf0 service nova] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:31.335834 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11508fc0-999a-4a2a-b7b0-013b66eadbf0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:31.344453 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11508fc0-999a-4a2a-b7b0-013b66eadbf0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:40:31.346218 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11508fc0-999a-4a2a-b7b0-013b66eadbf0 service nova] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:40:31.356153 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 760/1524] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:40:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:40:32.175888 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4343331c-6a2b-4459-8de8-1d3bd34cfd66 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:32.176453 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4343331c-6a2b-4459-8de8-1d3bd34cfd66 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:32.176453 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4343331c-6a2b-4459-8de8-1d3bd34cfd66 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:32.177780 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 765/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 14:40:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:40:32.182512 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf08ae-5873-4d33-a661-3092ae68dcd5 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:32.182945 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf08ae-5873-4d33-a661-3092ae68dcd5 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:32.183043 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf08ae-5873-4d33-a661-3092ae68dcd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:32.205981 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-d4bf08ae-5873-4d33-a661-3092ae68dcd5 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:40:32.207177 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-d4bf08ae-5873-4d33-a661-3092ae68dcd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:40:32.215040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 761/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 14:40:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:40:32.366543 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4343331c-6a2b-4459-8de8-1d3bd34cfd66 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:32.381198 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4343331c-6a2b-4459-8de8-1d3bd34cfd66 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:32.386525 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402a4abf-7ac7-4138-82a0-9b739c2692cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:32.387113 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a4abf-7ac7-4138-82a0-9b739c2692cd service nova] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:32.387423 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a4abf-7ac7-4138-82a0-9b739c2692cd service nova] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:32.387694 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a4abf-7ac7-4138-82a0-9b739c2692cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:32.396245 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402a4abf-7ac7-4138-82a0-9b739c2692cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:40:32.397811 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402a4abf-7ac7-4138-82a0-9b739c2692cd service nova] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:40:32.407131 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 766/1527] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:40:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:40:36.330319 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41fd3b1-3048-4ba6-9075-b1df76781b0c None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:36.330849 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41fd3b1-3048-4ba6-9075-b1df76781b0c None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:36.330849 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41fd3b1-3048-4ba6-9075-b1df76781b0c None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:36.332251 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 762/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 14:40:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:40:36.337356 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5c5020-782d-41e0-b7e8-e65f007aaa68 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:36.337766 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5c5020-782d-41e0-b7e8-e65f007aaa68 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:36.337899 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5c5020-782d-41e0-b7e8-e65f007aaa68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:36.357732 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-5c5c5020-782d-41e0-b7e8-e65f007aaa68 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:40:36.358931 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-5c5c5020-782d-41e0-b7e8-e65f007aaa68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:40:36.367552 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 767/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 14:40:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:40:36.659686 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41fd3b1-3048-4ba6-9075-b1df76781b0c None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:36.661389 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-b41fd3b1-3048-4ba6-9075-b1df76781b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:40:36.739357 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41fd3b1-3048-4ba6-9075-b1df76781b0c None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:36.742890 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60436616-4a03-4a67-b7fc-b549834cc990 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:36.743510 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60436616-4a03-4a67-b7fc-b549834cc990 service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:36.743787 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60436616-4a03-4a67-b7fc-b549834cc990 service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:36.744061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60436616-4a03-4a67-b7fc-b549834cc990 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:36.752866 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60436616-4a03-4a67-b7fc-b549834cc990 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:40:36.754284 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60436616-4a03-4a67-b7fc-b549834cc990 service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:40:36.763539 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 763/1530] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:40:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:40:38.669695 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96898368-0cc2-45ef-8cda-4f614d34bee9 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:38.670268 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96898368-0cc2-45ef-8cda-4f614d34bee9 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:38.670268 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96898368-0cc2-45ef-8cda-4f614d34bee9 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:38.671615 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 768/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 14:40:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:40:38.676932 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d8c5d-a8d9-42c2-80af-835d3a5709d9 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:38.677367 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d8c5d-a8d9-42c2-80af-835d3a5709d9 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:38.677495 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d8c5d-a8d9-42c2-80af-835d3a5709d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:38.697361 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-bd4d8c5d-a8d9-42c2-80af-835d3a5709d9 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:40:38.698570 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-bd4d8c5d-a8d9-42c2-80af-835d3a5709d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:40:38.707130 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 764/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 14:40:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:40:38.996991 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96898368-0cc2-45ef-8cda-4f614d34bee9 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:39.008331 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96898368-0cc2-45ef-8cda-4f614d34bee9 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:39.011844 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca47bf0-5e70-46de-b770-033af3a63805 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:39.012427 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca47bf0-5e70-46de-b770-033af3a63805 service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:39.012708 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca47bf0-5e70-46de-b770-033af3a63805 service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:39.012975 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca47bf0-5e70-46de-b770-033af3a63805 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:39.022460 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca47bf0-5e70-46de-b770-033af3a63805 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:40:39.023908 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca47bf0-5e70-46de-b770-033af3a63805 service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:40:39.033386 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 769/1533] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:40:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:40:42.414758 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233036d-429c-4cc5-90e1-64cdbd530759 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:42.415311 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233036d-429c-4cc5-90e1-64cdbd530759 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:42.415311 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233036d-429c-4cc5-90e1-64cdbd530759 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:42.416700 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 765/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 14:40:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:40:42.422240 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65933cc9-a483-45b0-9b10-19b47173cfa7 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:42.422687 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65933cc9-a483-45b0-9b10-19b47173cfa7 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:42.422792 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65933cc9-a483-45b0-9b10-19b47173cfa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:42.430334 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=81 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:40:42.444368 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-65933cc9-a483-45b0-9b10-19b47173cfa7 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:40:42.445567 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-65933cc9-a483-45b0-9b10-19b47173cfa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:40:42.453367 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 770/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 14:40:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:40:43.006912 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2233036d-429c-4cc5-90e1-64cdbd530759 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:43.008752 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-2233036d-429c-4cc5-90e1-64cdbd530759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:40:43.074094 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:40:43.087583 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2233036d-429c-4cc5-90e1-64cdbd530759 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:43.091195 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb24ade-872b-48df-9711-9b6e3d38ab78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:43.091821 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb24ade-872b-48df-9711-9b6e3d38ab78 service cinder] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:43.092101 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb24ade-872b-48df-9711-9b6e3d38ab78 service cinder] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:43.092372 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb24ade-872b-48df-9711-9b6e3d38ab78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:43.101287 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb24ade-872b-48df-9711-9b6e3d38ab78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:40:43.102699 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb24ade-872b-48df-9711-9b6e3d38ab78 service cinder] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:40:43.112629 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 766/1536] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:40:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:40:45.029943 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2a234-fb7e-4311-bce4-a9718fe67c84 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:45.032090 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2a234-fb7e-4311-bce4-a9718fe67c84 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:45.032090 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2a234-fb7e-4311-bce4-a9718fe67c84 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:45.032300 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 771/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 14:40:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:40:45.037463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c38c7-8233-4279-bf70-7684e4218a7b None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:45.037880 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c38c7-8233-4279-bf70-7684e4218a7b None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:45.038005 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c38c7-8233-4279-bf70-7684e4218a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:45.057544 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-126c38c7-8233-4279-bf70-7684e4218a7b None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:40:45.058730 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-126c38c7-8233-4279-bf70-7684e4218a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:40:45.067940 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 767/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 14:40:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:40:45.625793 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b2a234-fb7e-4311-bce4-a9718fe67c84 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:40:45.627496 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-26b2a234-fb7e-4311-bce4-a9718fe67c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:40:45.640205 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b2a234-fb7e-4311-bce4-a9718fe67c84 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:40:45.643675 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6897007-bd26-453e-b037-79510b5acce2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:40:45.644260 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6897007-bd26-453e-b037-79510b5acce2 service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:40:45.644537 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6897007-bd26-453e-b037-79510b5acce2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:40:45.644809 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6897007-bd26-453e-b037-79510b5acce2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:40:45.654003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6897007-bd26-453e-b037-79510b5acce2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:40:45.655493 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6897007-bd26-453e-b037-79510b5acce2 service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:40:45.665487 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 772/1539] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:40:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:40:55.052751 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=11 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:40:55.661211 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=8 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:09.174222 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c6a37-5ad0-4394-bd27-9aec7fb045ae None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:09.176008 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c6a37-5ad0-4394-bd27-9aec7fb045ae None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:09.176008 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c6a37-5ad0-4394-bd27-9aec7fb045ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:09.212517 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-b61c6a37-5ad0-4394-bd27-9aec7fb045ae None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:09.214038 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-b61c6a37-5ad0-4394-bd27-9aec7fb045ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:09.234028 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 768/1540] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:09.243399 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794e1ed-819c-43df-a1bd-13b0af2fa0db None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:09.244491 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794e1ed-819c-43df-a1bd-13b0af2fa0db None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:09.245927 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794e1ed-819c-43df-a1bd-13b0af2fa0db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:09.273960 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-1794e1ed-819c-43df-a1bd-13b0af2fa0db None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:09.275437 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-1794e1ed-819c-43df-a1bd-13b0af2fa0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:09.287947 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 773/1541] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:09.292589 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c6a37-5ad0-4394-bd27-9aec7fb045ae None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:09.304211 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c6a37-5ad0-4394-bd27-9aec7fb045ae None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:09.311956 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3dc03f-b658-4d56-b9cf-b5416c96b40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:09.313334 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3dc03f-b658-4d56-b9cf-b5416c96b40a admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:09.313703 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3dc03f-b658-4d56-b9cf-b5416c96b40a admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:09.313981 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3dc03f-b658-4d56-b9cf-b5416c96b40a admin admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:09.314648 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3dc03f-b658-4d56-b9cf-b5416c96b40a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:42:09.315225 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3dc03f-b658-4d56-b9cf-b5416c96b40a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:09.316858 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3dc03f-b658-4d56-b9cf-b5416c96b40a admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:09.337019 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 769/1542] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 14:42:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:42:09.345393 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1794e1ed-819c-43df-a1bd-13b0af2fa0db None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:09.355101 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1794e1ed-819c-43df-a1bd-13b0af2fa0db None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:09.362390 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01a9ec-3f69-41e4-afd7-85d4e754b7b1 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:09.362743 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01a9ec-3f69-41e4-afd7-85d4e754b7b1 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:09.362956 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01a9ec-3f69-41e4-afd7-85d4e754b7b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:09.372376 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ba11d-07d2-4e5b-a9d7-9b636104dbfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:09.373018 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ba11d-07d2-4e5b-a9d7-9b636104dbfa admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:09.373296 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ba11d-07d2-4e5b-a9d7-9b636104dbfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:09.373571 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ba11d-07d2-4e5b-a9d7-9b636104dbfa admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:09.374243 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9ba11d-07d2-4e5b-a9d7-9b636104dbfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:09.376899 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9ba11d-07d2-4e5b-a9d7-9b636104dbfa admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:09.409624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-9f01a9ec-3f69-41e4-afd7-85d4e754b7b1 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:09.411019 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-9f01a9ec-3f69-41e4-afd7-85d4e754b7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:09.430931 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 770/1543] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:09.432255 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada02e3-186d-412b-90a5-f20dc7b389c4 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:09.432545 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada02e3-186d-412b-90a5-f20dc7b389c4 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:09.432875 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada02e3-186d-412b-90a5-f20dc7b389c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:09.465448 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-8ada02e3-186d-412b-90a5-f20dc7b389c4 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:09.466703 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-8ada02e3-186d-412b-90a5-f20dc7b389c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:09.478857 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 774/1544] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:42:09 2022] POST /identity/v3/projects => generated 373 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:42:09.480002 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58193c0e-81c6-4bf9-be60-aec7709c8b30 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:09.480283 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58193c0e-81c6-4bf9-be60-aec7709c8b30 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:09.480550 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58193c0e-81c6-4bf9-be60-aec7709c8b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:09.520891 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-58193c0e-81c6-4bf9-be60-aec7709c8b30 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:09.523186 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-58193c0e-81c6-4bf9-be60-aec7709c8b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:09.586982 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 771/1545] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:09.588193 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ada02e3-186d-412b-90a5-f20dc7b389c4 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:09.610866 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ada02e3-186d-412b-90a5-f20dc7b389c4 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:09.628681 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515be887-90e2-4aae-8525-2804d353031f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:09.629308 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515be887-90e2-4aae-8525-2804d353031f admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:09.629584 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515be887-90e2-4aae-8525-2804d353031f admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:09.629851 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515be887-90e2-4aae-8525-2804d353031f admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:09.630489 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515be887-90e2-4aae-8525-2804d353031f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:09.635473 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515be887-90e2-4aae-8525-2804d353031f admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:09.655558 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 775/1546] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:09.657014 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b4a1bd-e681-4121-a014-c687642b09f5 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:09.657304 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b4a1bd-e681-4121-a014-c687642b09f5 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:09.657571 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b4a1bd-e681-4121-a014-c687642b09f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:09.679330 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 772/1547] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:42:09 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:09.683036 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748692f6-fe62-4e27-b555-7550ed21fc16 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:09.686043 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748692f6-fe62-4e27-b555-7550ed21fc16 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:09.686322 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748692f6-fe62-4e27-b555-7550ed21fc16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:09.700859 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-40b4a1bd-e681-4121-a014-c687642b09f5 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:09.704175 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-40b4a1bd-e681-4121-a014-c687642b09f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:09.723623 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 776/1548] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:09.724548 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b4a1bd-e681-4121-a014-c687642b09f5 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:09.734930 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-748692f6-fe62-4e27-b555-7550ed21fc16 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:09.736163 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-748692f6-fe62-4e27-b555-7550ed21fc16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:09.743908 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b4a1bd-e681-4121-a014-c687642b09f5 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:09.754690 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 773/1549] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:09.755590 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748692f6-fe62-4e27-b555-7550ed21fc16 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:09.756634 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb041eaf-453b-4372-a6ae-01248f674f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:09.757618 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb041eaf-453b-4372-a6ae-01248f674f28 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:09.757893 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb041eaf-453b-4372-a6ae-01248f674f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:09.758160 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb041eaf-453b-4372-a6ae-01248f674f28 admin admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:09.758818 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb041eaf-453b-4372-a6ae-01248f674f28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:42:09.759119 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb041eaf-453b-4372-a6ae-01248f674f28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:09.763511 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: WARNING py.warnings [None req-cb041eaf-453b-4372-a6ae-01248f674f28 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 30 14:42:09.763511 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: warnings.warn(msg) Aug 30 14:42:09.763511 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: Aug 30 14:42:09.764568 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb041eaf-453b-4372-a6ae-01248f674f28 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:09.774038 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748692f6-fe62-4e27-b555-7550ed21fc16 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:09.779179 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f06d64-6719-4522-85bf-2291f83b826d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:09.779815 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f06d64-6719-4522-85bf-2291f83b826d admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:09.780088 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f06d64-6719-4522-85bf-2291f83b826d admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:09.780354 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f06d64-6719-4522-85bf-2291f83b826d admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:09.781015 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f06d64-6719-4522-85bf-2291f83b826d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:09.783598 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f06d64-6719-4522-85bf-2291f83b826d admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:09.784525 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 777/1550] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 14:42:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:42:09.785662 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193100ca-9bc6-4b1b-9831-5d98bb182772 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:09.785989 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193100ca-9bc6-4b1b-9831-5d98bb182772 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:09.786263 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193100ca-9bc6-4b1b-9831-5d98bb182772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:09.798766 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 774/1551] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:09 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:09.802219 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f06d64-6719-4522-85bf-2291f83b826d admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:09.808915 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-193100ca-9bc6-4b1b-9831-5d98bb182772 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:09.810161 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-193100ca-9bc6-4b1b-9831-5d98bb182772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:09.813915 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f06d64-6719-4522-85bf-2291f83b826d admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:09.818621 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 778/1552] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:09.819920 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc92212-e191-4d41-bc63-7c14c1ed871b None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:09.820197 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc92212-e191-4d41-bc63-7c14c1ed871b None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:09.820478 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc92212-e191-4d41-bc63-7c14c1ed871b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:09.820733 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b3925c-a3fa-40a0-bc30-e349b4b17683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:09.821358 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b3925c-a3fa-40a0-bc30-e349b4b17683 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:09.821659 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b3925c-a3fa-40a0-bc30-e349b4b17683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:09.822221 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b3925c-a3fa-40a0-bc30-e349b4b17683 admin admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:09.822699 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b3925c-a3fa-40a0-bc30-e349b4b17683 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:42:09.823006 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b3925c-a3fa-40a0-bc30-e349b4b17683 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:09.825030 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b3925c-a3fa-40a0-bc30-e349b4b17683 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:09.838975 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 775/1553] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 14:42:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:42:09.840275 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-6cc92212-e191-4d41-bc63-7c14c1ed871b None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:09.840479 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509725d-d423-4a0c-a516-43e62cc2c7af None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:09.840756 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509725d-d423-4a0c-a516-43e62cc2c7af None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:09.841039 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509725d-d423-4a0c-a516-43e62cc2c7af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:09.841458 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-6cc92212-e191-4d41-bc63-7c14c1ed871b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:09.850139 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 779/1554] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:09.851372 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db1a09-b57f-45b8-9de4-29c114b9d992 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:09.851605 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db1a09-b57f-45b8-9de4-29c114b9d992 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:09.851873 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db1a09-b57f-45b8-9de4-29c114b9d992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:09.864011 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-4509725d-d423-4a0c-a516-43e62cc2c7af None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:09.865229 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-4509725d-d423-4a0c-a516-43e62cc2c7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:09.872067 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-15db1a09-b57f-45b8-9de4-29c114b9d992 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:09.873207 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-15db1a09-b57f-45b8-9de4-29c114b9d992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:09.873659 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 776/1555] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:09.874445 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4509725d-d423-4a0c-a516-43e62cc2c7af None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:09.884076 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4509725d-d423-4a0c-a516-43e62cc2c7af None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:09.887254 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 780/1556] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:09.888056 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15db1a09-b57f-45b8-9de4-29c114b9d992 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:09.888842 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f20b1-8e9a-4f07-966d-1c488c91b622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:09.889455 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f20b1-8e9a-4f07-966d-1c488c91b622 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:09.889734 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f20b1-8e9a-4f07-966d-1c488c91b622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:09.890005 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f20b1-8e9a-4f07-966d-1c488c91b622 admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:09.891174 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456f20b1-8e9a-4f07-966d-1c488c91b622 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:09.892234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: WARNING py.warnings [None req-456f20b1-8e9a-4f07-966d-1c488c91b622 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 14:42:09.892234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: warnings.warn(msg) Aug 30 14:42:09.892234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: Aug 30 14:42:09.893779 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456f20b1-8e9a-4f07-966d-1c488c91b622 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:09.898504 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15db1a09-b57f-45b8-9de4-29c114b9d992 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:09.903130 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4c6601-637d-4476-8d36-eecbfe06da16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:09.903746 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c6601-637d-4476-8d36-eecbfe06da16 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:09.904022 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c6601-637d-4476-8d36-eecbfe06da16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:09.904290 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c6601-637d-4476-8d36-eecbfe06da16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:09.904927 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4c6601-637d-4476-8d36-eecbfe06da16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:09.907105 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4c6601-637d-4476-8d36-eecbfe06da16 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:09.919115 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 781/1557] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:09.920343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc4d6a-2f60-40cb-ad46-ba0d44b7701b None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:09.920685 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc4d6a-2f60-40cb-ad46-ba0d44b7701b None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:09.920889 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc4d6a-2f60-40cb-ad46-ba0d44b7701b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:09.927693 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 777/1558] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:42:09 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:09.927693 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f20b1-8e9a-4f07-966d-1c488c91b622 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:09.940670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-6dbc4d6a-2f60-40cb-ad46-ba0d44b7701b None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:09.941777 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-6dbc4d6a-2f60-40cb-ad46-ba0d44b7701b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:09.986039 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:09.989743 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f20b1-8e9a-4f07-966d-1c488c91b622 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:09.996079 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0499480c-b5e9-46c2-bdab-d06a885b9635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:09.996681 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499480c-b5e9-46c2-bdab-d06a885b9635 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:09.996961 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499480c-b5e9-46c2-bdab-d06a885b9635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:09.997225 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499480c-b5e9-46c2-bdab-d06a885b9635 admin admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:09.997865 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0499480c-b5e9-46c2-bdab-d06a885b9635 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:42:09.998287 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0499480c-b5e9-46c2-bdab-d06a885b9635 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:10.000976 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0499480c-b5e9-46c2-bdab-d06a885b9635 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:10.004947 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 782/1559] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:10.005897 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbc4d6a-2f60-40cb-ad46-ba0d44b7701b None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:10.016688 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 778/1560] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 14:42:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:42:10.016987 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbc4d6a-2f60-40cb-ad46-ba0d44b7701b None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:10.017931 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47dae03-9351-41c8-accc-266a49728190 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:10.018231 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47dae03-9351-41c8-accc-266a49728190 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:10.018479 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47dae03-9351-41c8-accc-266a49728190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:10.021950 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd813494-a453-47fd-85a5-e537670cb696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:10.022598 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd813494-a453-47fd-85a5-e537670cb696 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:10.022877 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd813494-a453-47fd-85a5-e537670cb696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:10.023145 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd813494-a453-47fd-85a5-e537670cb696 admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:10.023900 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd813494-a453-47fd-85a5-e537670cb696 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:10.025884 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd813494-a453-47fd-85a5-e537670cb696 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:10.038322 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:10.038511 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-d47dae03-9351-41c8-accc-266a49728190 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:10.039669 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-d47dae03-9351-41c8-accc-266a49728190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:10.048706 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 779/1561] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:10.049722 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e7f9c-d59c-49c5-b201-26079bedfe07 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:10.050001 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e7f9c-d59c-49c5-b201-26079bedfe07 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:10.050272 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e7f9c-d59c-49c5-b201-26079bedfe07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:10.063748 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 783/1562] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:42:10 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:10.063748 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e9ec2-e1ab-4818-9ba5-e07f363bd722 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:10.063748 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e9ec2-e1ab-4818-9ba5-e07f363bd722 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:10.063748 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e9ec2-e1ab-4818-9ba5-e07f363bd722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:10.070643 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-077e7f9c-d59c-49c5-b201-26079bedfe07 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:10.071871 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-077e7f9c-d59c-49c5-b201-26079bedfe07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:10.082822 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-ae4e9ec2-e1ab-4818-9ba5-e07f363bd722 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:10.084016 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-ae4e9ec2-e1ab-4818-9ba5-e07f363bd722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:10.133356 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 780/1563] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:10.134371 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077e7f9c-d59c-49c5-b201-26079bedfe07 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:10.143954 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077e7f9c-d59c-49c5-b201-26079bedfe07 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:10.147327 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 784/1564] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:10.148261 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e9ec2-e1ab-4818-9ba5-e07f363bd722 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:10.149029 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d2b1ee-4995-45b6-90a2-1adaf3bcec35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:10.149740 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2b1ee-4995-45b6-90a2-1adaf3bcec35 admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:10.150022 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2b1ee-4995-45b6-90a2-1adaf3bcec35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:10.150291 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2b1ee-4995-45b6-90a2-1adaf3bcec35 admin admin] PATH_INFO: `/v3/projects/a9044e72cc1e4f869bf08d6ccc8b6a38/users/f9b316ccd13c4e35aaf24761eb7965c9/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:10.157311 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e9ec2-e1ab-4818-9ba5-e07f363bd722 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:10.161997 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be28f88-1abb-42c1-9823-8de3a9e1eb33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:10.162666 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be28f88-1abb-42c1-9823-8de3a9e1eb33 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:10.162948 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be28f88-1abb-42c1-9823-8de3a9e1eb33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:10.163233 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be28f88-1abb-42c1-9823-8de3a9e1eb33 admin admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:10.163870 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be28f88-1abb-42c1-9823-8de3a9e1eb33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:10.164899 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: WARNING py.warnings [None req-5be28f88-1abb-42c1-9823-8de3a9e1eb33 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 14:42:10.164899 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: warnings.warn(msg) Aug 30 14:42:10.164899 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: Aug 30 14:42:10.166162 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be28f88-1abb-42c1-9823-8de3a9e1eb33 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:10.166521 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d2b1ee-4995-45b6-90a2-1adaf3bcec35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9044e72cc1e4f869bf08d6ccc8b6a38, user_id=f9b316ccd13c4e35aaf24761eb7965c9, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:10.167595 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: WARNING py.warnings [None req-42d2b1ee-4995-45b6-90a2-1adaf3bcec35 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 14:42:10.167595 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: warnings.warn(msg) Aug 30 14:42:10.167595 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: Aug 30 14:42:10.169532 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d2b1ee-4995-45b6-90a2-1adaf3bcec35 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:10.184080 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 781/1565] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:42:10 2022] PUT /identity/v3/projects/a9044e72cc1e4f869bf08d6ccc8b6a38/users/f9b316ccd13c4e35aaf24761eb7965c9/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:10.186017 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c271266-6273-4a28-9cd4-70f86d721819 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:10.186319 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c271266-6273-4a28-9cd4-70f86d721819 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:10.186570 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c271266-6273-4a28-9cd4-70f86d721819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:10.202595 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 785/1566] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:42:10 2022] POST /identity/v3/users => generated 290 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:10.204330 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be28f88-1abb-42c1-9823-8de3a9e1eb33 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:10.209603 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-6c271266-6273-4a28-9cd4-70f86d721819 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:10.209603 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-6c271266-6273-4a28-9cd4-70f86d721819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:10.214156 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be28f88-1abb-42c1-9823-8de3a9e1eb33 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:10.270637 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3c8220-1efd-4b96-985d-35e89b066e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:10.271369 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3c8220-1efd-4b96-985d-35e89b066e92 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:10.271569 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3c8220-1efd-4b96-985d-35e89b066e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:10.271953 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3c8220-1efd-4b96-985d-35e89b066e92 admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:10.272594 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3c8220-1efd-4b96-985d-35e89b066e92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:10.272864 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 782/1567] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:10.274504 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3c8220-1efd-4b96-985d-35e89b066e92 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:10.276078 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c271266-6273-4a28-9cd4-70f86d721819 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:10.285718 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c271266-6273-4a28-9cd4-70f86d721819 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:10.290309 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af142728-84d5-4079-ac0f-a07e1580cb65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:10.291103 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af142728-84d5-4079-ac0f-a07e1580cb65 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:10.291355 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af142728-84d5-4079-ac0f-a07e1580cb65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:10.291655 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af142728-84d5-4079-ac0f-a07e1580cb65 admin admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:10.292650 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af142728-84d5-4079-ac0f-a07e1580cb65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:42:10.293107 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af142728-84d5-4079-ac0f-a07e1580cb65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:10.295106 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af142728-84d5-4079-ac0f-a07e1580cb65 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:10.304210 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 786/1568] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:42:10 2022] POST /identity/v3/projects => generated 367 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:10.304861 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3c8220-1efd-4b96-985d-35e89b066e92 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:10.318260 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 783/1569] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 14:42:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:42:10.318260 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af142728-84d5-4079-ac0f-a07e1580cb65 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:10.327677 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af142728-84d5-4079-ac0f-a07e1580cb65 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:10.373692 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:10.374346 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3c8220-1efd-4b96-985d-35e89b066e92 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:10.380395 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc6afc1-03a5-4243-9285-82e0900bf92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:10.381134 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc6afc1-03a5-4243-9285-82e0900bf92f admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:10.381417 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc6afc1-03a5-4243-9285-82e0900bf92f admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:10.381688 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc6afc1-03a5-4243-9285-82e0900bf92f admin admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:10.382315 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc6afc1-03a5-4243-9285-82e0900bf92f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:10.384272 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc6afc1-03a5-4243-9285-82e0900bf92f admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:10.387384 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de30df5-d158-47c2-902d-ae22d4e5a1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:10.387997 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de30df5-d158-47c2-902d-ae22d4e5a1c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:10.388298 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de30df5-d158-47c2-902d-ae22d4e5a1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:10.388552 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de30df5-d158-47c2-902d-ae22d4e5a1c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:10.389222 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de30df5-d158-47c2-902d-ae22d4e5a1c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:42:10.389525 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de30df5-d158-47c2-902d-ae22d4e5a1c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:10.391176 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de30df5-d158-47c2-902d-ae22d4e5a1c2 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:10.392715 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:10.404443 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 784/1570] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 14:42:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:42:10.405279 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de30df5-d158-47c2-902d-ae22d4e5a1c2 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:10.416538 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de30df5-d158-47c2-902d-ae22d4e5a1c2 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:10.416878 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 787/1571] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:42:10 2022] POST /identity/v3/users => generated 287 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:10.418233 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0534c89-2ba7-46ca-aac4-6d80fbda29b6 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:10.418518 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0534c89-2ba7-46ca-aac4-6d80fbda29b6 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:10.418789 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0534c89-2ba7-46ca-aac4-6d80fbda29b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:10.424577 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d6026-1b81-4983-9f99-b45eb3fffd48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:10.425184 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d6026-1b81-4983-9f99-b45eb3fffd48 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:10.425465 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d6026-1b81-4983-9f99-b45eb3fffd48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:10.425741 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d6026-1b81-4983-9f99-b45eb3fffd48 admin admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:10.426384 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2d6026-1b81-4983-9f99-b45eb3fffd48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:42:10.426683 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2d6026-1b81-4983-9f99-b45eb3fffd48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:10.428337 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2d6026-1b81-4983-9f99-b45eb3fffd48 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:10.447969 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 785/1572] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 14:42:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:42:10.447969 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d6026-1b81-4983-9f99-b45eb3fffd48 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:10.451502 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-a0534c89-2ba7-46ca-aac4-6d80fbda29b6 None None] MFA Rules not processed for user `f9b316ccd13c4e35aaf24761eb7965c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:10.452751 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-a0534c89-2ba7-46ca-aac4-6d80fbda29b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:10.453025 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d6026-1b81-4983-9f99-b45eb3fffd48 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:10.457802 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d93007-eed1-4ede-9f23-b6338a30ac6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:10.458403 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d93007-eed1-4ede-9f23-b6338a30ac6b admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:10.458680 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d93007-eed1-4ede-9f23-b6338a30ac6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:10.458955 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d93007-eed1-4ede-9f23-b6338a30ac6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:10.459621 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d93007-eed1-4ede-9f23-b6338a30ac6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:10.461214 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d93007-eed1-4ede-9f23-b6338a30ac6b admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:10.472179 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 786/1573] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:10.472988 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d93007-eed1-4ede-9f23-b6338a30ac6b admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:10.484962 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d93007-eed1-4ede-9f23-b6338a30ac6b admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:10.489580 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de42a9-dbc3-41a7-8766-2991217f894e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:10.490165 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de42a9-dbc3-41a7-8766-2991217f894e admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:10.490443 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de42a9-dbc3-41a7-8766-2991217f894e admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:10.490712 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de42a9-dbc3-41a7-8766-2991217f894e admin admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:10.491452 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6de42a9-dbc3-41a7-8766-2991217f894e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:42:10.491747 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6de42a9-dbc3-41a7-8766-2991217f894e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:10.493329 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6de42a9-dbc3-41a7-8766-2991217f894e admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:10.512282 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 787/1574] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 14:42:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:42:10.512282 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de42a9-dbc3-41a7-8766-2991217f894e admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:10.515635 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de42a9-dbc3-41a7-8766-2991217f894e admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:10.520228 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8346ca3-f2eb-4ea0-8419-e5fff4f25ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:10.520811 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8346ca3-f2eb-4ea0-8419-e5fff4f25ab9 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:10.521089 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8346ca3-f2eb-4ea0-8419-e5fff4f25ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:10.521364 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8346ca3-f2eb-4ea0-8419-e5fff4f25ab9 admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:10.522003 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8346ca3-f2eb-4ea0-8419-e5fff4f25ab9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:10.524011 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8346ca3-f2eb-4ea0-8419-e5fff4f25ab9 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:10.527607 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 788/1575] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:10.528426 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0534c89-2ba7-46ca-aac4-6d80fbda29b6 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:10.539851 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0534c89-2ba7-46ca-aac4-6d80fbda29b6 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:10.544722 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a8a83-4e4b-49d5-9dfb-18a52011259e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:10.545319 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a8a83-4e4b-49d5-9dfb-18a52011259e admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:10.545600 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a8a83-4e4b-49d5-9dfb-18a52011259e admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:10.545873 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a8a83-4e4b-49d5-9dfb-18a52011259e admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:10.545995 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 788/1576] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:42:10 2022] POST /identity/v3/users => generated 284 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:10.546504 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955a8a83-4e4b-49d5-9dfb-18a52011259e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:10.547230 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8346ca3-f2eb-4ea0-8419-e5fff4f25ab9 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:10.549095 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955a8a83-4e4b-49d5-9dfb-18a52011259e admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:10.557464 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8346ca3-f2eb-4ea0-8419-e5fff4f25ab9 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:10.562105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228067c5-3daf-4edd-87cb-c865b1c3dd54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:10.563337 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228067c5-3daf-4edd-87cb-c865b1c3dd54 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:10.563624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228067c5-3daf-4edd-87cb-c865b1c3dd54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:10.563898 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228067c5-3daf-4edd-87cb-c865b1c3dd54 admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:10.564529 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228067c5-3daf-4edd-87cb-c865b1c3dd54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:10.566687 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228067c5-3daf-4edd-87cb-c865b1c3dd54 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:10.578149 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 789/1577] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:42:10 2022] POST /identity/v3/projects => generated 391 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:10.579301 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a8a83-4e4b-49d5-9dfb-18a52011259e admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:10.602116 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 789/1578] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:42:10 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:10.602116 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228067c5-3daf-4edd-87cb-c865b1c3dd54 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:10.644609 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a8a83-4e4b-49d5-9dfb-18a52011259e admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:10.666305 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228067c5-3daf-4edd-87cb-c865b1c3dd54 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:10.671117 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5606ad92-f117-473a-9fa6-c00db570dca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:10.672287 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5606ad92-f117-473a-9fa6-c00db570dca4 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:10.672558 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5606ad92-f117-473a-9fa6-c00db570dca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:10.672847 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5606ad92-f117-473a-9fa6-c00db570dca4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:10.673473 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5606ad92-f117-473a-9fa6-c00db570dca4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:10.675503 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5606ad92-f117-473a-9fa6-c00db570dca4 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:10.703604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 790/1579] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:42:10 2022] POST /identity/v3/projects => generated 375 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:10.704419 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5606ad92-f117-473a-9fa6-c00db570dca4 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:10.704619 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66024f26-911d-42a8-b9e3-1451c85c1a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:10.705176 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66024f26-911d-42a8-b9e3-1451c85c1a12 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:10.705460 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66024f26-911d-42a8-b9e3-1451c85c1a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:10.705795 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66024f26-911d-42a8-b9e3-1451c85c1a12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:10.706610 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66024f26-911d-42a8-b9e3-1451c85c1a12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:10.708288 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66024f26-911d-42a8-b9e3-1451c85c1a12 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:10.713578 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:10.720339 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 790/1580] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:10 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:10.721880 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66024f26-911d-42a8-b9e3-1451c85c1a12 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:10.732849 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66024f26-911d-42a8-b9e3-1451c85c1a12 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:10.758047 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:10.771054 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5606ad92-f117-473a-9fa6-c00db570dca4 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:10.775979 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3767cf-0834-4af9-b7cf-3e5caea73277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:10.776588 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3767cf-0834-4af9-b7cf-3e5caea73277 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:10.776894 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3767cf-0834-4af9-b7cf-3e5caea73277 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:10.777164 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3767cf-0834-4af9-b7cf-3e5caea73277 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:10.777796 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3767cf-0834-4af9-b7cf-3e5caea73277 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:10.779412 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3767cf-0834-4af9-b7cf-3e5caea73277 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:10.789809 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 791/1581] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:10 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:10.790659 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3767cf-0834-4af9-b7cf-3e5caea73277 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:10.791099 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad5241-eb04-41c6-b42c-d408df93d488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:10.791854 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad5241-eb04-41c6-b42c-d408df93d488 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:10.792132 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad5241-eb04-41c6-b42c-d408df93d488 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:10.792401 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad5241-eb04-41c6-b42c-d408df93d488 admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:10.793026 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ad5241-eb04-41c6-b42c-d408df93d488 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:10.794922 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ad5241-eb04-41c6-b42c-d408df93d488 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:10.800728 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3767cf-0834-4af9-b7cf-3e5caea73277 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:10.805414 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3846e5-eb44-43db-aff1-0c5feac7d120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:10.806020 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3846e5-eb44-43db-aff1-0c5feac7d120 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:10.806300 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3846e5-eb44-43db-aff1-0c5feac7d120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:10.806604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3846e5-eb44-43db-aff1-0c5feac7d120 admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:10.807277 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3846e5-eb44-43db-aff1-0c5feac7d120 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:10.809188 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3846e5-eb44-43db-aff1-0c5feac7d120 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:10.823883 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 791/1582] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:42:10 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:10.824755 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad5241-eb04-41c6-b42c-d408df93d488 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:10.839915 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 792/1583] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:42:10 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:10.840929 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3846e5-eb44-43db-aff1-0c5feac7d120 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:10.888683 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad5241-eb04-41c6-b42c-d408df93d488 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:10.901024 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3846e5-eb44-43db-aff1-0c5feac7d120 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:10.905610 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa14e2d7-2415-4535-806b-78322e549744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:10.906718 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa14e2d7-2415-4535-806b-78322e549744 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:10.907001 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa14e2d7-2415-4535-806b-78322e549744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:10.907295 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa14e2d7-2415-4535-806b-78322e549744 admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:10.907952 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa14e2d7-2415-4535-806b-78322e549744 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:10.909910 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa14e2d7-2415-4535-806b-78322e549744 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:10.931319 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 793/1584] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:42:10 2022] POST /identity/v3/users => generated 296 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:10.932006 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa14e2d7-2415-4535-806b-78322e549744 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:10.941875 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa14e2d7-2415-4535-806b-78322e549744 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:10.946601 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f2fcea-2499-4a5f-babb-1c5bfd58120b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:10.947186 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2fcea-2499-4a5f-babb-1c5bfd58120b admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:10.947496 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2fcea-2499-4a5f-babb-1c5bfd58120b admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:10.947766 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2fcea-2499-4a5f-babb-1c5bfd58120b admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:10.948395 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f2fcea-2499-4a5f-babb-1c5bfd58120b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:10.950323 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f2fcea-2499-4a5f-babb-1c5bfd58120b admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:10.950582 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ff343a-7d29-47c8-aeb4-ef041308cf25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:10.951195 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff343a-7d29-47c8-aeb4-ef041308cf25 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:10.951505 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff343a-7d29-47c8-aeb4-ef041308cf25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:10.951779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff343a-7d29-47c8-aeb4-ef041308cf25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:10.952423 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ff343a-7d29-47c8-aeb4-ef041308cf25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:10.954215 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ff343a-7d29-47c8-aeb4-ef041308cf25 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:10.964088 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 792/1585] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:10 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:10.964888 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ff343a-7d29-47c8-aeb4-ef041308cf25 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:10.975355 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 794/1586] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:42:10 2022] POST /identity/v3/users => generated 287 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:10.975808 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ff343a-7d29-47c8-aeb4-ef041308cf25 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:10.978631 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f2fcea-2499-4a5f-babb-1c5bfd58120b admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:10.980558 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d5a46-d5bd-4fa2-9903-5d062e8b7dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:10.981152 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d5a46-d5bd-4fa2-9903-5d062e8b7dcf admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:10.981430 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d5a46-d5bd-4fa2-9903-5d062e8b7dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:10.981701 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d5a46-d5bd-4fa2-9903-5d062e8b7dcf admin admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:10.982324 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922d5a46-d5bd-4fa2-9903-5d062e8b7dcf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:10.985013 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922d5a46-d5bd-4fa2-9903-5d062e8b7dcf admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:10.988679 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f2fcea-2499-4a5f-babb-1c5bfd58120b admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:10.993331 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8190c431-1e95-48ed-80ea-829d1841e344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:10.993945 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190c431-1e95-48ed-80ea-829d1841e344 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:10.994229 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190c431-1e95-48ed-80ea-829d1841e344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:10.994506 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190c431-1e95-48ed-80ea-829d1841e344 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:10.995444 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8190c431-1e95-48ed-80ea-829d1841e344 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:10.996794 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8190c431-1e95-48ed-80ea-829d1841e344 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:11.006463 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 793/1587] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:42:10 2022] POST /identity/v3/users => generated 288 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:11.007302 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d5a46-d5bd-4fa2-9903-5d062e8b7dcf admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:11.007863 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 795/1588] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:11.008528 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8190c431-1e95-48ed-80ea-829d1841e344 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:11.017728 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d5a46-d5bd-4fa2-9903-5d062e8b7dcf admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:11.018381 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8190c431-1e95-48ed-80ea-829d1841e344 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:11.022492 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c48fd-32f5-44f1-9941-33b4e2b255c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:11.022921 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a151bb-d5f0-41a9-b69b-f4abbaa4a7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:11.023370 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c48fd-32f5-44f1-9941-33b4e2b255c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:11.023534 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a151bb-d5f0-41a9-b69b-f4abbaa4a7d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:11.023653 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c48fd-32f5-44f1-9941-33b4e2b255c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:11.023817 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a151bb-d5f0-41a9-b69b-f4abbaa4a7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:11.023932 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c48fd-32f5-44f1-9941-33b4e2b255c0 admin admin] PATH_INFO: `/v3/projects/8aeffb40eafc44639c8e5032dce345aa/users/0642fa92a86c4e01a42bf087cd36cace/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:11.024086 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a151bb-d5f0-41a9-b69b-f4abbaa4a7d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:11.024710 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a151bb-d5f0-41a9-b69b-f4abbaa4a7d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:11.026597 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a151bb-d5f0-41a9-b69b-f4abbaa4a7d8 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:11.033095 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:11.041774 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c48fd-32f5-44f1-9941-33b4e2b255c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8aeffb40eafc44639c8e5032dce345aa, user_id=0642fa92a86c4e01a42bf087cd36cace, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:11.044567 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c48fd-32f5-44f1-9941-33b4e2b255c0 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:11.049191 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 796/1589] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:42:11 2022] POST /identity/v3/users => generated 294 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:11.050097 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a151bb-d5f0-41a9-b69b-f4abbaa4a7d8 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:11.057666 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 794/1590] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:42:11 2022] PUT /identity/v3/projects/8aeffb40eafc44639c8e5032dce345aa/users/0642fa92a86c4e01a42bf087cd36cace/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:11.058548 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c48fd-32f5-44f1-9941-33b4e2b255c0 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:11.063048 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a151bb-d5f0-41a9-b69b-f4abbaa4a7d8 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:11.120246 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d881544-7221-45a8-9f2e-2982c22951de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:11.120862 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d881544-7221-45a8-9f2e-2982c22951de admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:11.121169 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d881544-7221-45a8-9f2e-2982c22951de admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:11.121441 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d881544-7221-45a8-9f2e-2982c22951de admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:11.121640 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c48fd-32f5-44f1-9941-33b4e2b255c0 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:11.122068 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d881544-7221-45a8-9f2e-2982c22951de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:11.124079 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d881544-7221-45a8-9f2e-2982c22951de admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:11.126326 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c01f7a-d15e-43bd-9a96-d93ed01339a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:11.126914 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c01f7a-d15e-43bd-9a96-d93ed01339a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:11.127189 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c01f7a-d15e-43bd-9a96-d93ed01339a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:11.127485 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c01f7a-d15e-43bd-9a96-d93ed01339a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:11.128119 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c01f7a-d15e-43bd-9a96-d93ed01339a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:11.129685 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c01f7a-d15e-43bd-9a96-d93ed01339a1 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:11.140293 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=87,keystone:INSERT=13 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:11.141695 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 795/1591] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:11 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:11.142265 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c01f7a-d15e-43bd-9a96-d93ed01339a1 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:11.151743 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 797/1592] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:42:11 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:11.152613 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d881544-7221-45a8-9f2e-2982c22951de admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:11.153017 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c01f7a-d15e-43bd-9a96-d93ed01339a1 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:11.158200 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117d78fa-0e84-4c88-8318-6d450c1bf2c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:11.158792 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d78fa-0e84-4c88-8318-6d450c1bf2c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:11.159073 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d78fa-0e84-4c88-8318-6d450c1bf2c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:11.159368 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d78fa-0e84-4c88-8318-6d450c1bf2c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:11.160020 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117d78fa-0e84-4c88-8318-6d450c1bf2c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:11.161671 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117d78fa-0e84-4c88-8318-6d450c1bf2c5 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:11.163013 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d881544-7221-45a8-9f2e-2982c22951de admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:11.168043 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab10f29-a390-4ef3-afcb-a7183a3d587d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:11.168645 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab10f29-a390-4ef3-afcb-a7183a3d587d admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:11.168928 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab10f29-a390-4ef3-afcb-a7183a3d587d admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:11.169254 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab10f29-a390-4ef3-afcb-a7183a3d587d admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:11.169921 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab10f29-a390-4ef3-afcb-a7183a3d587d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:11.172525 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab10f29-a390-4ef3-afcb-a7183a3d587d admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:11.172664 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 796/1593] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:11.172664 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117d78fa-0e84-4c88-8318-6d450c1bf2c5 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:11.183134 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 798/1594] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:11.183984 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab10f29-a390-4ef3-afcb-a7183a3d587d admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:11.185281 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117d78fa-0e84-4c88-8318-6d450c1bf2c5 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:11.190340 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db98e4ca-2064-4d4d-be6d-31478f66b4b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:11.190984 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db98e4ca-2064-4d4d-be6d-31478f66b4b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:11.191286 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db98e4ca-2064-4d4d-be6d-31478f66b4b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:11.191563 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db98e4ca-2064-4d4d-be6d-31478f66b4b1 admin admin] PATH_INFO: `/v3/projects/6e9991d4feb042f19745ffb1d9c8b0cf/users/1754147b3a494d76be36555c7f9542db/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:11.195542 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab10f29-a390-4ef3-afcb-a7183a3d587d admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:11.200222 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bfc97b-9b67-425a-80fa-79ad83dd28f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:11.200809 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bfc97b-9b67-425a-80fa-79ad83dd28f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:11.201089 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bfc97b-9b67-425a-80fa-79ad83dd28f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:11.201375 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bfc97b-9b67-425a-80fa-79ad83dd28f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:11.202001 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bfc97b-9b67-425a-80fa-79ad83dd28f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:11.204143 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bfc97b-9b67-425a-80fa-79ad83dd28f9 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:11.208202 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db98e4ca-2064-4d4d-be6d-31478f66b4b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e9991d4feb042f19745ffb1d9c8b0cf, user_id=1754147b3a494d76be36555c7f9542db, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:11.211555 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db98e4ca-2064-4d4d-be6d-31478f66b4b1 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:11.221661 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 799/1595] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:11 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:11.222436 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bfc97b-9b67-425a-80fa-79ad83dd28f9 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:11.231706 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 797/1596] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:42:11 2022] PUT /identity/v3/projects/6e9991d4feb042f19745ffb1d9c8b0cf/users/1754147b3a494d76be36555c7f9542db/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:11.238989 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44dc026-5e81-4647-87a3-604e843d0269 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:11.238989 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44dc026-5e81-4647-87a3-604e843d0269 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:11.238989 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44dc026-5e81-4647-87a3-604e843d0269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:11.239301 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bfc97b-9b67-425a-80fa-79ad83dd28f9 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:11.255391 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-b44dc026-5e81-4647-87a3-604e843d0269 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:11.256662 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-b44dc026-5e81-4647-87a3-604e843d0269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:11.294874 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f9ebc1-caa4-4b2f-ba2d-4eadb8ff86c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:11.295545 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f9ebc1-caa4-4b2f-ba2d-4eadb8ff86c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:11.295825 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f9ebc1-caa4-4b2f-ba2d-4eadb8ff86c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:11.296096 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f9ebc1-caa4-4b2f-ba2d-4eadb8ff86c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:11.296739 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f9ebc1-caa4-4b2f-ba2d-4eadb8ff86c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:11.298482 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f9ebc1-caa4-4b2f-ba2d-4eadb8ff86c1 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:11.309128 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 800/1597] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:11 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:11.309763 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f9ebc1-caa4-4b2f-ba2d-4eadb8ff86c1 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:11.318239 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 798/1598] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:11.318993 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44dc026-5e81-4647-87a3-604e843d0269 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:11.322630 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f9ebc1-caa4-4b2f-ba2d-4eadb8ff86c1 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:11.327532 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed13b5c-88a7-4595-8055-1f95744afbdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:11.328821 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44dc026-5e81-4647-87a3-604e843d0269 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:11.328968 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed13b5c-88a7-4595-8055-1f95744afbdd admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:11.329098 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed13b5c-88a7-4595-8055-1f95744afbdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:11.329369 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed13b5c-88a7-4595-8055-1f95744afbdd admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:11.330116 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed13b5c-88a7-4595-8055-1f95744afbdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:11.331869 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed13b5c-88a7-4595-8055-1f95744afbdd admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:11.333946 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2377c5d3-b77f-48ed-a882-a396868b3f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:11.334143 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377c5d3-b77f-48ed-a882-a396868b3f71 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:11.334406 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377c5d3-b77f-48ed-a882-a396868b3f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:11.334677 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377c5d3-b77f-48ed-a882-a396868b3f71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:11.335548 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2377c5d3-b77f-48ed-a882-a396868b3f71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:11.337041 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2377c5d3-b77f-48ed-a882-a396868b3f71 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:11.345279 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 801/1599] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:11.346160 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed13b5c-88a7-4595-8055-1f95744afbdd admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:11.350677 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 799/1600] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:11.351554 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2377c5d3-b77f-48ed-a882-a396868b3f71 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:11.356156 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed13b5c-88a7-4595-8055-1f95744afbdd admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:11.361455 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81fd41f-1ab5-4478-8316-5d1b7c8cffa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:11.361837 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2377c5d3-b77f-48ed-a882-a396868b3f71 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:11.362101 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81fd41f-1ab5-4478-8316-5d1b7c8cffa2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:11.362378 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81fd41f-1ab5-4478-8316-5d1b7c8cffa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:11.362650 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81fd41f-1ab5-4478-8316-5d1b7c8cffa2 admin admin] PATH_INFO: `/v3/projects/9db62f1e868a420680e55a9c81d79817/users/dad848db58cb45a8b0a9b0f74b9632ed/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:11.366715 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424bcdd9-51b9-4b40-a2e6-0910b6d3308c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:11.367533 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424bcdd9-51b9-4b40-a2e6-0910b6d3308c admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:11.367864 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424bcdd9-51b9-4b40-a2e6-0910b6d3308c admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:11.368212 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424bcdd9-51b9-4b40-a2e6-0910b6d3308c admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:11.368902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424bcdd9-51b9-4b40-a2e6-0910b6d3308c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:11.370819 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424bcdd9-51b9-4b40-a2e6-0910b6d3308c admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:11.378057 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81fd41f-1ab5-4478-8316-5d1b7c8cffa2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9db62f1e868a420680e55a9c81d79817, user_id=dad848db58cb45a8b0a9b0f74b9632ed, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:11.380788 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81fd41f-1ab5-4478-8316-5d1b7c8cffa2 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:11.381492 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 800/1601] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:11.384100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424bcdd9-51b9-4b40-a2e6-0910b6d3308c admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:11.394990 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 802/1602] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:42:11 2022] PUT /identity/v3/projects/9db62f1e868a420680e55a9c81d79817/users/dad848db58cb45a8b0a9b0f74b9632ed/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:11.395408 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424bcdd9-51b9-4b40-a2e6-0910b6d3308c admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:11.396279 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8399f-c30b-4102-a088-42a67b219a97 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:11.396619 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8399f-c30b-4102-a088-42a67b219a97 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:11.396907 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8399f-c30b-4102-a088-42a67b219a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:11.426622 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-dde8399f-c30b-4102-a088-42a67b219a97 None None] MFA Rules not processed for user `1754147b3a494d76be36555c7f9542db`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:11.428075 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-dde8399f-c30b-4102-a088-42a67b219a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:11.431814 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:11.466072 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce018ce-cb43-4aac-8a9c-0464f1e5ebb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:11.466595 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce018ce-cb43-4aac-8a9c-0464f1e5ebb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:11.466874 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce018ce-cb43-4aac-8a9c-0464f1e5ebb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:11.467150 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce018ce-cb43-4aac-8a9c-0464f1e5ebb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:11.467819 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce018ce-cb43-4aac-8a9c-0464f1e5ebb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:11.469481 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce018ce-cb43-4aac-8a9c-0464f1e5ebb2 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:11.479754 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 801/1603] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:11 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:11.480560 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce018ce-cb43-4aac-8a9c-0464f1e5ebb2 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:11.491102 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce018ce-cb43-4aac-8a9c-0464f1e5ebb2 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:11.495472 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 803/1604] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:11.495735 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c71f0a3-17f9-47d2-98dc-51d277fca645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:11.496315 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c71f0a3-17f9-47d2-98dc-51d277fca645 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:11.496434 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8399f-c30b-4102-a088-42a67b219a97 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:11.496595 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c71f0a3-17f9-47d2-98dc-51d277fca645 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:11.496870 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c71f0a3-17f9-47d2-98dc-51d277fca645 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:11.497501 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c71f0a3-17f9-47d2-98dc-51d277fca645 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:11.498113 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-dde8399f-c30b-4102-a088-42a67b219a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:11.499250 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c71f0a3-17f9-47d2-98dc-51d277fca645 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:11.510704 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 802/1605] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:11.511553 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c71f0a3-17f9-47d2-98dc-51d277fca645 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:11.523088 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c71f0a3-17f9-47d2-98dc-51d277fca645 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:11.527744 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e15870b-dde4-4df1-a4ff-c6008dd88736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:11.528394 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e15870b-dde4-4df1-a4ff-c6008dd88736 admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:11.528667 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e15870b-dde4-4df1-a4ff-c6008dd88736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:11.528943 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e15870b-dde4-4df1-a4ff-c6008dd88736 admin admin] PATH_INFO: `/v3/projects/65bab38256b44ad6b4b573a2441eb6d3/users/35f28006916040c3b988de6c15093166/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:11.545693 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:11.548414 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e15870b-dde4-4df1-a4ff-c6008dd88736 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65bab38256b44ad6b4b573a2441eb6d3, user_id=35f28006916040c3b988de6c15093166, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:11.551312 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e15870b-dde4-4df1-a4ff-c6008dd88736 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:11.564520 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 803/1606] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:42:11 2022] PUT /identity/v3/projects/65bab38256b44ad6b4b573a2441eb6d3/users/35f28006916040c3b988de6c15093166/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:11.565582 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e15870b-dde4-4df1-a4ff-c6008dd88736 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:11.579163 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8399f-c30b-4102-a088-42a67b219a97 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:11.627601 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e15870b-dde4-4df1-a4ff-c6008dd88736 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:11.628602 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f3288b-562d-4073-b2b7-a7bcddd1d0d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:11.629581 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3288b-562d-4073-b2b7-a7bcddd1d0d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:11.630241 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3288b-562d-4073-b2b7-a7bcddd1d0d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:11.630576 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3288b-562d-4073-b2b7-a7bcddd1d0d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:11.632432 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7eb3d1-8f3f-4468-a148-a6396c715858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:11.633086 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7eb3d1-8f3f-4468-a148-a6396c715858 admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:11.633423 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7eb3d1-8f3f-4468-a148-a6396c715858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:11.633632 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7eb3d1-8f3f-4468-a148-a6396c715858 admin admin] PATH_INFO: `/v3/projects/9273d5abc65f4a45bb302211f5e69a28/users/a4d4dc90f4b44e69bddb822b3f5f577d/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:11.640024 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f3288b-562d-4073-b2b7-a7bcddd1d0d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:11.641688 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f3288b-562d-4073-b2b7-a7bcddd1d0d7 service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:11.648654 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7eb3d1-8f3f-4468-a148-a6396c715858 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9273d5abc65f4a45bb302211f5e69a28, user_id=a4d4dc90f4b44e69bddb822b3f5f577d, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:11.651376 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7eb3d1-8f3f-4468-a148-a6396c715858 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:11.651533 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 804/1607] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:11.652374 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f3288b-562d-4073-b2b7-a7bcddd1d0d7 service neutron] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:11.663656 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f3288b-562d-4073-b2b7-a7bcddd1d0d7 service neutron] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:11.667031 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 804/1608] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:42:11 2022] PUT /identity/v3/projects/9273d5abc65f4a45bb302211f5e69a28/users/a4d4dc90f4b44e69bddb822b3f5f577d/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:11.671133 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa5e8b-0a63-409f-b332-23bf0393b0f1 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:11.671676 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa5e8b-0a63-409f-b332-23bf0393b0f1 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:11.673104 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa5e8b-0a63-409f-b332-23bf0393b0f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:11.704823 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-3bfa5e8b-0a63-409f-b332-23bf0393b0f1 None None] MFA Rules not processed for user `dad848db58cb45a8b0a9b0f74b9632ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:11.706329 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-3bfa5e8b-0a63-409f-b332-23bf0393b0f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:11.735085 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d72b7c6-dc9b-4588-ae95-c420b0ca7184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:11.735846 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72b7c6-dc9b-4588-ae95-c420b0ca7184 admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:11.736224 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72b7c6-dc9b-4588-ae95-c420b0ca7184 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:11.736573 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72b7c6-dc9b-4588-ae95-c420b0ca7184 admin admin] PATH_INFO: `/v3/projects/b6e19883ffdc44498f95e13a1338ed10/users/dd4c29b19bae4df9b88607c43aa17aa8/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:11.752916 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d72b7c6-dc9b-4588-ae95-c420b0ca7184 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6e19883ffdc44498f95e13a1338ed10, user_id=dd4c29b19bae4df9b88607c43aa17aa8, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:11.755683 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d72b7c6-dc9b-4588-ae95-c420b0ca7184 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:11.771498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 805/1609] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:42:11 2022] PUT /identity/v3/projects/b6e19883ffdc44498f95e13a1338ed10/users/dd4c29b19bae4df9b88607c43aa17aa8/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:11.772463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d72b7c6-dc9b-4588-ae95-c420b0ca7184 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:11.798688 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:11.820450 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 805/1610] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:11.822696 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa5e8b-0a63-409f-b332-23bf0393b0f1 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:11.841039 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa5e8b-0a63-409f-b332-23bf0393b0f1 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:11.871583 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d72b7c6-dc9b-4588-ae95-c420b0ca7184 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:11.877803 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f4d63-a052-47dc-830b-d72f2e663cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:11.878679 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f4d63-a052-47dc-830b-d72f2e663cc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:11.879153 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f4d63-a052-47dc-830b-d72f2e663cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:11.879582 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f4d63-a052-47dc-830b-d72f2e663cc6 admin admin] PATH_INFO: `/v3/projects/04efb141e084405992bfc0418a74da2a/users/ca9bafc5ff4e43719f37e62cad946cf7/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:11.895942 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:11.899638 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574f4d63-a052-47dc-830b-d72f2e663cc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04efb141e084405992bfc0418a74da2a, user_id=ca9bafc5ff4e43719f37e62cad946cf7, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:11.902979 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574f4d63-a052-47dc-830b-d72f2e663cc6 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:11.912874 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3455f11f-9b40-48ec-a8f4-9864163ad5f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:11.913458 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455f11f-9b40-48ec-a8f4-9864163ad5f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:11.913707 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455f11f-9b40-48ec-a8f4-9864163ad5f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:11.913987 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455f11f-9b40-48ec-a8f4-9864163ad5f8 admin admin] PATH_INFO: `/v3/projects/a8914c5bc86f4ee6a9797b05a6a7b12b/users/6270e19d6245453ab79555a79532faba/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:11.925069 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 806/1611] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:42:11 2022] PUT /identity/v3/projects/04efb141e084405992bfc0418a74da2a/users/ca9bafc5ff4e43719f37e62cad946cf7/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:11.928143 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a14c97-ed4a-4813-a34d-54968f3d8f82 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:11.928143 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a14c97-ed4a-4813-a34d-54968f3d8f82 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:11.928143 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a14c97-ed4a-4813-a34d-54968f3d8f82 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:11.929057 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 807/1612] 10.222.0.30 () {54 vars in 962 bytes} [Tue Aug 30 14:42:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:42:11.930113 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cdf6ee-c21c-4445-8059-426d9afe8be4 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:11.930394 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cdf6ee-c21c-4445-8059-426d9afe8be4 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:11.930666 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cdf6ee-c21c-4445-8059-426d9afe8be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:11.940781 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3455f11f-9b40-48ec-a8f4-9864163ad5f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8914c5bc86f4ee6a9797b05a6a7b12b, user_id=6270e19d6245453ab79555a79532faba, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:11.944643 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3455f11f-9b40-48ec-a8f4-9864163ad5f8 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:11.953577 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-51cdf6ee-c21c-4445-8059-426d9afe8be4 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:11.954780 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-51cdf6ee-c21c-4445-8059-426d9afe8be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:11.975677 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 806/1613] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:42:11 2022] PUT /identity/v3/projects/a8914c5bc86f4ee6a9797b05a6a7b12b/users/6270e19d6245453ab79555a79532faba/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:11.978432 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3455f11f-9b40-48ec-a8f4-9864163ad5f8 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:12.035126 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 808/1614] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:12.038330 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f6189-528c-4577-a552-107aaab2ae15 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:12.038616 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f6189-528c-4577-a552-107aaab2ae15 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:12.039122 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f6189-528c-4577-a552-107aaab2ae15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:12.067165 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3455f11f-9b40-48ec-a8f4-9864163ad5f8 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:12.072495 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5650662-eab4-457c-bd29-b0115c911cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:12.073399 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5650662-eab4-457c-bd29-b0115c911cf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:12.073830 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5650662-eab4-457c-bd29-b0115c911cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:12.074254 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5650662-eab4-457c-bd29-b0115c911cf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:12.075019 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5650662-eab4-457c-bd29-b0115c911cf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:12.076811 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5650662-eab4-457c-bd29-b0115c911cf2 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:12.088701 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 807/1615] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:11 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:12.089721 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5650662-eab4-457c-bd29-b0115c911cf2 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:12.090063 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-1a5f6189-528c-4577-a552-107aaab2ae15 None None] MFA Rules not processed for user `dd4c29b19bae4df9b88607c43aa17aa8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:12.092196 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-1a5f6189-528c-4577-a552-107aaab2ae15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:12.153381 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5650662-eab4-457c-bd29-b0115c911cf2 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:12.157922 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38342e1-78bb-4841-bab4-d7ec5b715715 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:12.158607 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38342e1-78bb-4841-bab4-d7ec5b715715 service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:12.158955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38342e1-78bb-4841-bab4-d7ec5b715715 service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:12.159334 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38342e1-78bb-4841-bab4-d7ec5b715715 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:12.174290 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 809/1616] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:12.176033 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096eea27-9959-4096-808f-aade8c194cd0 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:12.176464 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096eea27-9959-4096-808f-aade8c194cd0 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:12.176904 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096eea27-9959-4096-808f-aade8c194cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:12.196501 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-096eea27-9959-4096-808f-aade8c194cd0 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:12.197522 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:12.197691 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-096eea27-9959-4096-808f-aade8c194cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:12.223688 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 810/1617] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:12.223688 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f345e2cf-b229-4608-bd2f-37986226ee98 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:12.223688 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f345e2cf-b229-4608-bd2f-37986226ee98 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:12.223688 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f345e2cf-b229-4608-bd2f-37986226ee98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:12.241703 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38342e1-78bb-4841-bab4-d7ec5b715715 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:12.243408 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38342e1-78bb-4841-bab4-d7ec5b715715 service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:12.255969 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 808/1618] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:12.257543 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6347a090-12fc-40ee-a49f-a31d05856246 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:12.257982 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6347a090-12fc-40ee-a49f-a31d05856246 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:12.258434 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6347a090-12fc-40ee-a49f-a31d05856246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:12.262914 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-f345e2cf-b229-4608-bd2f-37986226ee98 None None] MFA Rules not processed for user `8878de66a03044e38e507028853a0cd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:12.264312 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-f345e2cf-b229-4608-bd2f-37986226ee98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:12.283047 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-6347a090-12fc-40ee-a49f-a31d05856246 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:12.285071 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-6347a090-12fc-40ee-a49f-a31d05856246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:12.297667 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 809/1619] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:12.304796 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6347a090-12fc-40ee-a49f-a31d05856246 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:12.323900 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:12.351916 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 811/1620] 10.222.0.30 () {58 vars in 1059 bytes} [Tue Aug 30 14:42:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:12.352760 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f345e2cf-b229-4608-bd2f-37986226ee98 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:12.363479 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f345e2cf-b229-4608-bd2f-37986226ee98 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:12.368642 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b1ec60-945f-4706-9daf-2cda014408ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:12.369468 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b1ec60-945f-4706-9daf-2cda014408ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:12.369766 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b1ec60-945f-4706-9daf-2cda014408ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:12.370057 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b1ec60-945f-4706-9daf-2cda014408ee admin admin] PATH_INFO: `/v3/projects/9273d5abc65f4a45bb302211f5e69a28/users/a4d4dc90f4b44e69bddb822b3f5f577d/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:12.372865 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b1ec60-945f-4706-9daf-2cda014408ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=9273d5abc65f4a45bb302211f5e69a28, user_id=a4d4dc90f4b44e69bddb822b3f5f577d, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:12.377456 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b1ec60-945f-4706-9daf-2cda014408ee admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:12.382292 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6347a090-12fc-40ee-a49f-a31d05856246 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:12.390376 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 812/1621] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:42:12 2022] PUT /identity/v3/projects/9273d5abc65f4a45bb302211f5e69a28/users/a4d4dc90f4b44e69bddb822b3f5f577d/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:12.399743 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0010db5e-9c11-47ee-a7ec-400d8fc048a8 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:12.399743 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0010db5e-9c11-47ee-a7ec-400d8fc048a8 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:12.399743 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0010db5e-9c11-47ee-a7ec-400d8fc048a8 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:12.399743 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 813/1622] 10.222.0.30 () {54 vars in 962 bytes} [Tue Aug 30 14:42:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:42:12.399743 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0010db5e-9c11-47ee-a7ec-400d8fc048a8 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:12.507774 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0010db5e-9c11-47ee-a7ec-400d8fc048a8 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:12.509841 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9431c28-8c6a-4b3f-ae04-4b36437255ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:12.510454 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9431c28-8c6a-4b3f-ae04-4b36437255ac service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:12.510738 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9431c28-8c6a-4b3f-ae04-4b36437255ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:12.511010 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9431c28-8c6a-4b3f-ae04-4b36437255ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:12.512029 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e88ee2-226a-40bb-83ff-76a693259f78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:12.512700 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e88ee2-226a-40bb-83ff-76a693259f78 service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:12.513032 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e88ee2-226a-40bb-83ff-76a693259f78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:12.513308 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e88ee2-226a-40bb-83ff-76a693259f78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:12.616226 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9431c28-8c6a-4b3f-ae04-4b36437255ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:12.617854 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e88ee2-226a-40bb-83ff-76a693259f78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:12.617979 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9431c28-8c6a-4b3f-ae04-4b36437255ac service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:12.618871 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e88ee2-226a-40bb-83ff-76a693259f78 service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:12.629683 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 810/1623] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:12.630792 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9431c28-8c6a-4b3f-ae04-4b36437255ac service neutron] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:12.633950 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 814/1624] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:12.635378 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e88ee2-226a-40bb-83ff-76a693259f78 service neutron] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:12.641702 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9431c28-8c6a-4b3f-ae04-4b36437255ac service neutron] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:12.646718 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0523aea4-e5b6-4de7-8baf-78d2650c8556 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:12.647372 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0523aea4-e5b6-4de7-8baf-78d2650c8556 service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:12.648510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0523aea4-e5b6-4de7-8baf-78d2650c8556 service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:12.648799 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0523aea4-e5b6-4de7-8baf-78d2650c8556 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:12.666588 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0523aea4-e5b6-4de7-8baf-78d2650c8556 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:12.668110 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0523aea4-e5b6-4de7-8baf-78d2650c8556 service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:12.691594 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 811/1625] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:12.692793 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345be60c-68c9-41ca-93dd-76cce7afe033 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:12.693098 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345be60c-68c9-41ca-93dd-76cce7afe033 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:12.693366 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345be60c-68c9-41ca-93dd-76cce7afe033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:12.739888 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e88ee2-226a-40bb-83ff-76a693259f78 service neutron] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:12.742211 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-345be60c-68c9-41ca-93dd-76cce7afe033 None None] MFA Rules not processed for user `8878de66a03044e38e507028853a0cd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:12.743489 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-345be60c-68c9-41ca-93dd-76cce7afe033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:12.747045 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae15b305-1868-4682-aa1d-99413e8c47fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:12.747845 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae15b305-1868-4682-aa1d-99413e8c47fd service nova] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:12.748348 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae15b305-1868-4682-aa1d-99413e8c47fd service nova] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:12.748720 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae15b305-1868-4682-aa1d-99413e8c47fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:12.766079 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 812/1626] 10.222.0.30 () {58 vars in 1059 bytes} [Tue Aug 30 14:42:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:12.767394 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ca431-6854-4d8c-8c7b-4973b584b725 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:12.767678 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ca431-6854-4d8c-8c7b-4973b584b725 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:12.767946 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ca431-6854-4d8c-8c7b-4973b584b725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:12.788231 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:12.820931 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:12.830380 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-2f1ca431-6854-4d8c-8c7b-4973b584b725 None None] MFA Rules not processed for user `a4d4dc90f4b44e69bddb822b3f5f577d`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:12.831859 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-2f1ca431-6854-4d8c-8c7b-4973b584b725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:12.834278 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae15b305-1868-4682-aa1d-99413e8c47fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:12.836422 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae15b305-1868-4682-aa1d-99413e8c47fd service nova] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:12.863246 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 815/1627] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:12.866116 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae15b305-1868-4682-aa1d-99413e8c47fd service nova] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:12.896697 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae15b305-1868-4682-aa1d-99413e8c47fd service nova] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:12.915023 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8256a-86f2-45b9-b35e-2ab9d1ad05e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:12.918126 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8256a-86f2-45b9-b35e-2ab9d1ad05e7 service nova] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:12.919910 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8256a-86f2-45b9-b35e-2ab9d1ad05e7 service nova] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:12.920350 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8256a-86f2-45b9-b35e-2ab9d1ad05e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:12.989876 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 813/1628] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:12 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:13.004307 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ca431-6854-4d8c-8c7b-4973b584b725 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:13.087886 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa8256a-86f2-45b9-b35e-2ab9d1ad05e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:13.090446 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa8256a-86f2-45b9-b35e-2ab9d1ad05e7 service nova] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:13.118817 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 816/1629] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:13.137253 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ca431-6854-4d8c-8c7b-4973b584b725 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:13.142294 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafaff03-dc69-4f48-b5bf-a1a662286b2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:13.143047 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafaff03-dc69-4f48-b5bf-a1a662286b2e service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:13.143363 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafaff03-dc69-4f48-b5bf-a1a662286b2e service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:13.143641 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafaff03-dc69-4f48-b5bf-a1a662286b2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:13.154859 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafaff03-dc69-4f48-b5bf-a1a662286b2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:13.157316 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafaff03-dc69-4f48-b5bf-a1a662286b2e service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:13.182704 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 814/1630] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:13 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:42:13.321125 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbf82ce-4d23-4877-b601-ddee7ec99d00 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:13.321618 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbf82ce-4d23-4877-b601-ddee7ec99d00 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:13.321733 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbf82ce-4d23-4877-b601-ddee7ec99d00 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:13.323490 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 817/1631] 10.222.0.30 () {54 vars in 912 bytes} [Tue Aug 30 14:42:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:42:13.333107 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169a5646-a489-4b1e-af8f-46cdbeabaf21 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:13.333396 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169a5646-a489-4b1e-af8f-46cdbeabaf21 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:13.333673 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169a5646-a489-4b1e-af8f-46cdbeabaf21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:13.387048 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-169a5646-a489-4b1e-af8f-46cdbeabaf21 None None] MFA Rules not processed for user `8878de66a03044e38e507028853a0cd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:13.388336 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-169a5646-a489-4b1e-af8f-46cdbeabaf21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:13.418698 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 815/1632] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Aug 30 14:42:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:13.436780 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbf82ce-4d23-4877-b601-ddee7ec99d00 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:13.455376 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbf82ce-4d23-4877-b601-ddee7ec99d00 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:13.460392 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae70755e-577e-4cd6-a005-e2bbf6cb4b1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:13.461307 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae70755e-577e-4cd6-a005-e2bbf6cb4b1c service cinder] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:13.461805 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae70755e-577e-4cd6-a005-e2bbf6cb4b1c service cinder] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:13.462283 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae70755e-577e-4cd6-a005-e2bbf6cb4b1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:13.476907 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae70755e-577e-4cd6-a005-e2bbf6cb4b1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:13.480996 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae70755e-577e-4cd6-a005-e2bbf6cb4b1c service cinder] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:13.510251 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 818/1633] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:13.729900 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca545112-8e20-4d45-b581-aeccfe9456b3 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:13.730259 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca545112-8e20-4d45-b581-aeccfe9456b3 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:13.730606 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca545112-8e20-4d45-b581-aeccfe9456b3 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:13.732509 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 816/1634] 10.222.0.30 () {54 vars in 912 bytes} [Tue Aug 30 14:42:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:42:13.743560 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced6b7b-7011-40b2-9c6d-9d8232f7a8ea None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:13.743844 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced6b7b-7011-40b2-9c6d-9d8232f7a8ea None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:13.744114 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced6b7b-7011-40b2-9c6d-9d8232f7a8ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:13.772209 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-7ced6b7b-7011-40b2-9c6d-9d8232f7a8ea None None] MFA Rules not processed for user `8878de66a03044e38e507028853a0cd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:13.773817 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-7ced6b7b-7011-40b2-9c6d-9d8232f7a8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:13.787839 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 819/1635] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Aug 30 14:42:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:13.803872 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca545112-8e20-4d45-b581-aeccfe9456b3 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:13.819327 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca545112-8e20-4d45-b581-aeccfe9456b3 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:13.824421 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c40cf6-b880-474c-b0af-1be78e928fd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:13.825323 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c40cf6-b880-474c-b0af-1be78e928fd5 service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:13.825760 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c40cf6-b880-474c-b0af-1be78e928fd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:13.826039 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c40cf6-b880-474c-b0af-1be78e928fd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:13.841155 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c40cf6-b880-474c-b0af-1be78e928fd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:13.848782 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c40cf6-b880-474c-b0af-1be78e928fd5 service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:13.952545 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 817/1636] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:14.010070 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ced6b7b-7011-40b2-9c6d-9d8232f7a8ea None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:14.025978 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ced6b7b-7011-40b2-9c6d-9d8232f7a8ea None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:14.040888 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aba4ed-5d28-4e87-bdd2-83bab5c8342c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:14.041903 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aba4ed-5d28-4e87-bdd2-83bab5c8342c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:14.042003 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aba4ed-5d28-4e87-bdd2-83bab5c8342c admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:14.043304 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aba4ed-5d28-4e87-bdd2-83bab5c8342c admin admin] PATH_INFO: `/v3/users/f9b316ccd13c4e35aaf24761eb7965c9` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:14.045759 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aba4ed-5d28-4e87-bdd2-83bab5c8342c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9b316ccd13c4e35aaf24761eb7965c9)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:14.054996 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: WARNING py.warnings [None req-b6aba4ed-5d28-4e87-bdd2-83bab5c8342c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 14:42:14.054996 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: warnings.warn(msg) Aug 30 14:42:14.054996 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: Aug 30 14:42:14.054996 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aba4ed-5d28-4e87-bdd2-83bab5c8342c admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:14.133224 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c40cf6-b880-474c-b0af-1be78e928fd5 service cinder] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:14.147436 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c40cf6-b880-474c-b0af-1be78e928fd5 service cinder] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:14.160238 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572374ac-0d1e-4663-a022-88f108cd98bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:14.162021 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572374ac-0d1e-4663-a022-88f108cd98bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:14.163023 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572374ac-0d1e-4663-a022-88f108cd98bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:14.163924 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572374ac-0d1e-4663-a022-88f108cd98bb admin admin] PATH_INFO: `/v3/users/dd4c29b19bae4df9b88607c43aa17aa8` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:14.170154 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572374ac-0d1e-4663-a022-88f108cd98bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd4c29b19bae4df9b88607c43aa17aa8)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:14.189654 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572374ac-0d1e-4663-a022-88f108cd98bb admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:14.214964 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-b6aba4ed-5d28-4e87-bdd2-83bab5c8342c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9b316ccd13c4e35aaf24761eb7965c9'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:14.216456 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-b6aba4ed-5d28-4e87-bdd2-83bab5c8342c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9b316ccd13c4e35aaf24761eb7965c9'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:14.240930 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-b6aba4ed-5d28-4e87-bdd2-83bab5c8342c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9b316ccd13c4e35aaf24761eb7965c9'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:14.267594 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-b6aba4ed-5d28-4e87-bdd2-83bab5c8342c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9b316ccd13c4e35aaf24761eb7965c9'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:14.268957 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-b6aba4ed-5d28-4e87-bdd2-83bab5c8342c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9b316ccd13c4e35aaf24761eb7965c9'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:14.293363 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-572374ac-0d1e-4663-a022-88f108cd98bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd4c29b19bae4df9b88607c43aa17aa8'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:14.294730 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-572374ac-0d1e-4663-a022-88f108cd98bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd4c29b19bae4df9b88607c43aa17aa8'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:14.320026 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-572374ac-0d1e-4663-a022-88f108cd98bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd4c29b19bae4df9b88607c43aa17aa8'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:14.331549 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-572374ac-0d1e-4663-a022-88f108cd98bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd4c29b19bae4df9b88607c43aa17aa8'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:14.334368 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:14.354108 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-572374ac-0d1e-4663-a022-88f108cd98bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd4c29b19bae4df9b88607c43aa17aa8'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:14.355385 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 820/1637] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:42:14 2022] DELETE /identity/v3/users/f9b316ccd13c4e35aaf24761eb7965c9 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:14.362625 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e453c9-8ca2-4907-bfde-20d6f853a994 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:14.362964 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e453c9-8ca2-4907-bfde-20d6f853a994 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:14.363177 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e453c9-8ca2-4907-bfde-20d6f853a994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:14.364025 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 818/1638] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:42:14 2022] DELETE /identity/v3/users/dd4c29b19bae4df9b88607c43aa17aa8 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:42:14.369492 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4594f8-3f5e-4ee5-a697-0d0861e6d8e8 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:14.369953 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4594f8-3f5e-4ee5-a697-0d0861e6d8e8 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:14.370401 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4594f8-3f5e-4ee5-a697-0d0861e6d8e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:14.413300 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-75e453c9-8ca2-4907-bfde-20d6f853a994 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:14.414597 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-75e453c9-8ca2-4907-bfde-20d6f853a994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:14.439362 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-1f4594f8-3f5e-4ee5-a697-0d0861e6d8e8 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:14.443765 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-1f4594f8-3f5e-4ee5-a697-0d0861e6d8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:14.456434 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:14.574475 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 821/1639] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:14.588427 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 819/1640] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:14.599812 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e453c9-8ca2-4907-bfde-20d6f853a994 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:14.601254 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-75e453c9-8ca2-4907-bfde-20d6f853a994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:14.609224 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4594f8-3f5e-4ee5-a697-0d0861e6d8e8 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:14.611411 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-1f4594f8-3f5e-4ee5-a697-0d0861e6d8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:14.724346 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e453c9-8ca2-4907-bfde-20d6f853a994 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:14.727313 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4594f8-3f5e-4ee5-a697-0d0861e6d8e8 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:14.732960 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03958446-357a-443e-81b7-d482737a2692 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:14.733823 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03958446-357a-443e-81b7-d482737a2692 service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:14.734011 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03958446-357a-443e-81b7-d482737a2692 service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:14.734198 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03958446-357a-443e-81b7-d482737a2692 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:14.736570 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab5041e-b027-4424-aeb3-0543cf8633ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:14.737550 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab5041e-b027-4424-aeb3-0543cf8633ba service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:14.738115 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab5041e-b027-4424-aeb3-0543cf8633ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:14.738564 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab5041e-b027-4424-aeb3-0543cf8633ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:14.754968 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03958446-357a-443e-81b7-d482737a2692 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:14.756658 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03958446-357a-443e-81b7-d482737a2692 service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:14.761562 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab5041e-b027-4424-aeb3-0543cf8633ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:14.764445 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab5041e-b027-4424-aeb3-0543cf8633ba service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:14.790370 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 820/1641] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:14.793203 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 822/1642] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:16.453747 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03958446-357a-443e-81b7-d482737a2692 service neutron] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:16.455447 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-03958446-357a-443e-81b7-d482737a2692 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:16.489032 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03958446-357a-443e-81b7-d482737a2692 service neutron] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:16.495901 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab5041e-b027-4424-aeb3-0543cf8633ba service neutron] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:16.497731 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-7ab5041e-b027-4424-aeb3-0543cf8633ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:16.501420 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106cdb5-15f0-4590-9099-17bab01703f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:16.502081 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106cdb5-15f0-4590-9099-17bab01703f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:16.502360 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106cdb5-15f0-4590-9099-17bab01703f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:16.502633 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106cdb5-15f0-4590-9099-17bab01703f3 admin admin] PATH_INFO: `/v3/projects/a9044e72cc1e4f869bf08d6ccc8b6a38` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:16.504784 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7106cdb5-15f0-4590-9099-17bab01703f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9044e72cc1e4f869bf08d6ccc8b6a38)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:16.505808 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: WARNING py.warnings [None req-7106cdb5-15f0-4590-9099-17bab01703f3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 14:42:16.505808 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: warnings.warn(msg) Aug 30 14:42:16.505808 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: Aug 30 14:42:16.507065 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7106cdb5-15f0-4590-9099-17bab01703f3 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:16.527424 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab5041e-b027-4424-aeb3-0543cf8633ba service neutron] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:16.539852 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bfc299-65f2-49a8-9fb4-5478e6dd3e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:16.540722 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bfc299-65f2-49a8-9fb4-5478e6dd3e3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:16.541123 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bfc299-65f2-49a8-9fb4-5478e6dd3e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:16.541772 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bfc299-65f2-49a8-9fb4-5478e6dd3e3b admin admin] PATH_INFO: `/v3/projects/b6e19883ffdc44498f95e13a1338ed10` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:16.542964 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bfc299-65f2-49a8-9fb4-5478e6dd3e3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6e19883ffdc44498f95e13a1338ed10)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:16.545353 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bfc299-65f2-49a8-9fb4-5478e6dd3e3b admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:16.670052 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-f5bfc299-65f2-49a8-9fb4-5478e6dd3e3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6e19883ffdc44498f95e13a1338ed10', 'request_id': 'req-f5bfc299-65f2-49a8-9fb4-5478e6dd3e3b'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:16.682054 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-7106cdb5-15f0-4590-9099-17bab01703f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9044e72cc1e4f869bf08d6ccc8b6a38', 'request_id': 'req-7106cdb5-15f0-4590-9099-17bab01703f3'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:16.684076 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-f5bfc299-65f2-49a8-9fb4-5478e6dd3e3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6e19883ffdc44498f95e13a1338ed10', 'request_id': 'req-f5bfc299-65f2-49a8-9fb4-5478e6dd3e3b'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:16.693955 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-f5bfc299-65f2-49a8-9fb4-5478e6dd3e3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6e19883ffdc44498f95e13a1338ed10', 'request_id': 'req-f5bfc299-65f2-49a8-9fb4-5478e6dd3e3b'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:16.707165 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-7106cdb5-15f0-4590-9099-17bab01703f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9044e72cc1e4f869bf08d6ccc8b6a38', 'request_id': 'req-7106cdb5-15f0-4590-9099-17bab01703f3'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:16.725317 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-f5bfc299-65f2-49a8-9fb4-5478e6dd3e3b admin admin] The token cache is being invalidate because project b6e19883ffdc44498f95e13a1338ed10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:42:16.725790 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-f5bfc299-65f2-49a8-9fb4-5478e6dd3e3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:16.726752 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-f5bfc299-65f2-49a8-9fb4-5478e6dd3e3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:16.728638 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 823/1643] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:42:16 2022] DELETE /identity/v3/projects/b6e19883ffdc44498f95e13a1338ed10 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:42:16.733621 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0174fc2-e20d-4e91-bb0a-a11ce5a1f420 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:16.733621 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0174fc2-e20d-4e91-bb0a-a11ce5a1f420 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:16.734002 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-7106cdb5-15f0-4590-9099-17bab01703f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9044e72cc1e4f869bf08d6ccc8b6a38', 'request_id': 'req-7106cdb5-15f0-4590-9099-17bab01703f3'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:16.743575 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0174fc2-e20d-4e91-bb0a-a11ce5a1f420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:16.766611 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-7106cdb5-15f0-4590-9099-17bab01703f3 admin admin] The token cache is being invalidate because project a9044e72cc1e4f869bf08d6ccc8b6a38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:42:16.767122 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-7106cdb5-15f0-4590-9099-17bab01703f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:16.768110 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-7106cdb5-15f0-4590-9099-17bab01703f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:16.770263 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 821/1644] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:42:16 2022] DELETE /identity/v3/projects/a9044e72cc1e4f869bf08d6ccc8b6a38 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:16.775689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106cdb5-15f0-4590-9099-17bab01703f3 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:16.778571 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-7106cdb5-15f0-4590-9099-17bab01703f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:16.780550 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-d0174fc2-e20d-4e91-bb0a-a11ce5a1f420 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:16.783730 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-d0174fc2-e20d-4e91-bb0a-a11ce5a1f420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:16.838286 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:16.891239 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:16.922803 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106cdb5-15f0-4590-9099-17bab01703f3 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:16.927650 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 824/1645] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:16.935509 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c9b90-2123-44dc-8eb8-8f9fd4005b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:16.936003 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cb0077-cf60-4b61-9eea-544e945e45c4 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:16.936302 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cb0077-cf60-4b61-9eea-544e945e45c4 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:16.936673 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cb0077-cf60-4b61-9eea-544e945e45c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:16.938400 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c9b90-2123-44dc-8eb8-8f9fd4005b7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:16.939656 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c9b90-2123-44dc-8eb8-8f9fd4005b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:16.941252 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c9b90-2123-44dc-8eb8-8f9fd4005b7c admin admin] PATH_INFO: `/v3/users/a4d4dc90f4b44e69bddb822b3f5f577d` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:16.943791 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358c9b90-2123-44dc-8eb8-8f9fd4005b7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4d4dc90f4b44e69bddb822b3f5f577d)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:16.946632 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358c9b90-2123-44dc-8eb8-8f9fd4005b7c admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:16.970926 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-f1cb0077-cf60-4b61-9eea-544e945e45c4 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:16.973613 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-f1cb0077-cf60-4b61-9eea-544e945e45c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:16.992325 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 825/1646] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:16.998989 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cb0077-cf60-4b61-9eea-544e945e45c4 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:17.022186 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cb0077-cf60-4b61-9eea-544e945e45c4 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:17.031931 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-358c9b90-2123-44dc-8eb8-8f9fd4005b7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4d4dc90f4b44e69bddb822b3f5f577d'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:17.032357 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddad6b4-9116-4955-8a22-e910fba389e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:17.033084 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddad6b4-9116-4955-8a22-e910fba389e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:17.033253 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddad6b4-9116-4955-8a22-e910fba389e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:17.033517 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddad6b4-9116-4955-8a22-e910fba389e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:17.033971 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-358c9b90-2123-44dc-8eb8-8f9fd4005b7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4d4dc90f4b44e69bddb822b3f5f577d'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:17.036381 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddad6b4-9116-4955-8a22-e910fba389e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:42:17.040775 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddad6b4-9116-4955-8a22-e910fba389e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:17.043283 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddad6b4-9116-4955-8a22-e910fba389e5 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:17.050445 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-358c9b90-2123-44dc-8eb8-8f9fd4005b7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4d4dc90f4b44e69bddb822b3f5f577d'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:17.058522 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-358c9b90-2123-44dc-8eb8-8f9fd4005b7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4d4dc90f4b44e69bddb822b3f5f577d'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:17.072599 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 826/1647] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 14:42:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:42:17.074766 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddad6b4-9116-4955-8a22-e910fba389e5 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:17.078407 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-358c9b90-2123-44dc-8eb8-8f9fd4005b7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4d4dc90f4b44e69bddb822b3f5f577d'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:17.088038 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddad6b4-9116-4955-8a22-e910fba389e5 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:17.092143 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 822/1648] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:42:16 2022] DELETE /identity/v3/users/a4d4dc90f4b44e69bddb822b3f5f577d => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:42:17.093226 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26a4611-4874-4d3c-95e7-8c4ba017d86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:17.094879 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a4611-4874-4d3c-95e7-8c4ba017d86b admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:17.095261 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a4611-4874-4d3c-95e7-8c4ba017d86b admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:17.095549 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a4611-4874-4d3c-95e7-8c4ba017d86b admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:17.096834 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26a4611-4874-4d3c-95e7-8c4ba017d86b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:17.098780 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26a4611-4874-4d3c-95e7-8c4ba017d86b admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:17.142309 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 827/1649] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:42:17 2022] POST /identity/v3/projects => generated 403 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:17.146062 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c9b90-2123-44dc-8eb8-8f9fd4005b7c admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:17.147602 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-358c9b90-2123-44dc-8eb8-8f9fd4005b7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:17.219425 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c9b90-2123-44dc-8eb8-8f9fd4005b7c admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:17.273457 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:17.289530 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7ce2ad-d51d-4534-94e0-84348f7ec9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:17.290577 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ce2ad-d51d-4534-94e0-84348f7ec9e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:17.291158 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ce2ad-d51d-4534-94e0-84348f7ec9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:17.291424 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ce2ad-d51d-4534-94e0-84348f7ec9e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:17.292635 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7ce2ad-d51d-4534-94e0-84348f7ec9e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:17.294276 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7ce2ad-d51d-4534-94e0-84348f7ec9e5 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:17.337346 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 823/1650] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:42:17 2022] POST /identity/v3/users => generated 302 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:17.338412 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26a4611-4874-4d3c-95e7-8c4ba017d86b admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:17.358515 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26a4611-4874-4d3c-95e7-8c4ba017d86b admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:17.367281 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fdb979-07e2-4685-b79d-064e081e2f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:17.368332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fdb979-07e2-4685-b79d-064e081e2f2f admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:17.368762 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fdb979-07e2-4685-b79d-064e081e2f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:17.369181 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fdb979-07e2-4685-b79d-064e081e2f2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:17.370011 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fdb979-07e2-4685-b79d-064e081e2f2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:17.372175 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fdb979-07e2-4685-b79d-064e081e2f2f admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:17.402451 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7ce2ad-d51d-4534-94e0-84348f7ec9e5 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:17.404816 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 828/1651] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:17 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:17.405160 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-3e7ce2ad-d51d-4534-94e0-84348f7ec9e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:17.405845 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fdb979-07e2-4685-b79d-064e081e2f2f admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:17.418585 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fdb979-07e2-4685-b79d-064e081e2f2f admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:17.426114 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43048ff5-1465-41ae-b67e-aabe57dd09be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:17.426813 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43048ff5-1465-41ae-b67e-aabe57dd09be admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:17.427097 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43048ff5-1465-41ae-b67e-aabe57dd09be admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:17.427408 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43048ff5-1465-41ae-b67e-aabe57dd09be admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:17.428070 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43048ff5-1465-41ae-b67e-aabe57dd09be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:17.428363 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7ce2ad-d51d-4534-94e0-84348f7ec9e5 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:17.430716 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43048ff5-1465-41ae-b67e-aabe57dd09be admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:17.443928 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b6af00-158f-47a5-adb3-a3881620cb29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:17.447404 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6af00-158f-47a5-adb3-a3881620cb29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:17.447812 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6af00-158f-47a5-adb3-a3881620cb29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:17.448197 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6af00-158f-47a5-adb3-a3881620cb29 admin admin] PATH_INFO: `/v3/users/1754147b3a494d76be36555c7f9542db` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:17.449222 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 829/1652] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:17 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:17.450033 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b6af00-158f-47a5-adb3-a3881620cb29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1754147b3a494d76be36555c7f9542db)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:17.450737 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43048ff5-1465-41ae-b67e-aabe57dd09be admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:17.457890 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b6af00-158f-47a5-adb3-a3881620cb29 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:17.462685 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43048ff5-1465-41ae-b67e-aabe57dd09be admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:17.468255 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db9a94-a736-4818-b95c-d9fd1199bc4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:17.468981 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db9a94-a736-4818-b95c-d9fd1199bc4e admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:17.469330 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db9a94-a736-4818-b95c-d9fd1199bc4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:17.469617 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db9a94-a736-4818-b95c-d9fd1199bc4e admin admin] PATH_INFO: `/v3/projects/96d3213bbb054cdf82301bd361be3c36/users/aafc618b52ae496299fdf53d2cac771a/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:17.514841 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4db9a94-a736-4818-b95c-d9fd1199bc4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=96d3213bbb054cdf82301bd361be3c36, user_id=aafc618b52ae496299fdf53d2cac771a, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:17.521955 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4db9a94-a736-4818-b95c-d9fd1199bc4e admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:17.559467 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 830/1653] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:42:17 2022] PUT /identity/v3/projects/96d3213bbb054cdf82301bd361be3c36/users/aafc618b52ae496299fdf53d2cac771a/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:17.570854 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d96d14-89f9-4bb1-9d0f-d956b1279bbe None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:17.571161 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d96d14-89f9-4bb1-9d0f-d956b1279bbe None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:17.571818 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d96d14-89f9-4bb1-9d0f-d956b1279bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:17.574011 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-e1b6af00-158f-47a5-adb3-a3881620cb29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1754147b3a494d76be36555c7f9542db'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:17.576631 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-e1b6af00-158f-47a5-adb3-a3881620cb29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1754147b3a494d76be36555c7f9542db'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:17.601288 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-e1b6af00-158f-47a5-adb3-a3881620cb29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1754147b3a494d76be36555c7f9542db'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:17.627487 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-e1b6af00-158f-47a5-adb3-a3881620cb29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1754147b3a494d76be36555c7f9542db'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:17.644985 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-e1b6af00-158f-47a5-adb3-a3881620cb29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1754147b3a494d76be36555c7f9542db'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:17.653668 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-31d96d14-89f9-4bb1-9d0f-d956b1279bbe None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:17.656714 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-31d96d14-89f9-4bb1-9d0f-d956b1279bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:17.665716 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 824/1654] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:42:17 2022] DELETE /identity/v3/users/1754147b3a494d76be36555c7f9542db => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:17.667064 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d339419-39d8-4768-a266-92d7015e2e6e None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:17.667376 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d339419-39d8-4768-a266-92d7015e2e6e None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:17.667652 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d339419-39d8-4768-a266-92d7015e2e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:17.716696 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-4d339419-39d8-4768-a266-92d7015e2e6e None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:17.719784 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-4d339419-39d8-4768-a266-92d7015e2e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:17.842303 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 831/1655] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:17.879370 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 825/1656] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:18.059155 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d96d14-89f9-4bb1-9d0f-d956b1279bbe None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:18.064061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-31d96d14-89f9-4bb1-9d0f-d956b1279bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:18.115576 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d339419-39d8-4768-a266-92d7015e2e6e None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:18.116999 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-4d339419-39d8-4768-a266-92d7015e2e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:18.274525 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d96d14-89f9-4bb1-9d0f-d956b1279bbe None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:18.287566 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:18.334180 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:18.347524 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d339419-39d8-4768-a266-92d7015e2e6e None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:18.361078 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1e4d80-8f04-4fca-9729-297500994609 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:18.363512 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e4d80-8f04-4fca-9729-297500994609 service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:18.364379 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e4d80-8f04-4fca-9729-297500994609 service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:18.377458 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e4d80-8f04-4fca-9729-297500994609 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:18.388734 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c2626-8d63-410f-bfa9-949e2f0518f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:18.389397 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c2626-8d63-410f-bfa9-949e2f0518f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:18.389666 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c2626-8d63-410f-bfa9-949e2f0518f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:18.389941 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c2626-8d63-410f-bfa9-949e2f0518f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:18.390277 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1e4d80-8f04-4fca-9729-297500994609 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:18.392015 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1e4d80-8f04-4fca-9729-297500994609 service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:18.421846 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 826/1657] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:18.435944 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3c2626-8d63-410f-bfa9-949e2f0518f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:18.438185 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3c2626-8d63-410f-bfa9-949e2f0518f9 service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:18.473353 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 832/1658] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:19.085820 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1e4d80-8f04-4fca-9729-297500994609 service neutron] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:19.087405 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-aa1e4d80-8f04-4fca-9729-297500994609 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:19.118202 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1e4d80-8f04-4fca-9729-297500994609 service neutron] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:19.137931 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2813d0e-edeb-4177-b5e1-8fbefade700e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:19.138858 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2813d0e-edeb-4177-b5e1-8fbefade700e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:19.139388 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2813d0e-edeb-4177-b5e1-8fbefade700e admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:19.139879 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2813d0e-edeb-4177-b5e1-8fbefade700e admin admin] PATH_INFO: `/v3/projects/9273d5abc65f4a45bb302211f5e69a28` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:19.143141 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2813d0e-edeb-4177-b5e1-8fbefade700e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9273d5abc65f4a45bb302211f5e69a28)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:19.146038 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2813d0e-edeb-4177-b5e1-8fbefade700e admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:19.356509 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-a2813d0e-edeb-4177-b5e1-8fbefade700e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9273d5abc65f4a45bb302211f5e69a28', 'request_id': 'req-a2813d0e-edeb-4177-b5e1-8fbefade700e'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:19.383697 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-a2813d0e-edeb-4177-b5e1-8fbefade700e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9273d5abc65f4a45bb302211f5e69a28', 'request_id': 'req-a2813d0e-edeb-4177-b5e1-8fbefade700e'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:19.394566 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-a2813d0e-edeb-4177-b5e1-8fbefade700e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9273d5abc65f4a45bb302211f5e69a28', 'request_id': 'req-a2813d0e-edeb-4177-b5e1-8fbefade700e'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:19.456421 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-a2813d0e-edeb-4177-b5e1-8fbefade700e admin admin] The token cache is being invalidate because project 9273d5abc65f4a45bb302211f5e69a28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:42:19.460067 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-a2813d0e-edeb-4177-b5e1-8fbefade700e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:19.462992 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-a2813d0e-edeb-4177-b5e1-8fbefade700e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:19.467119 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 827/1659] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:42:19 2022] DELETE /identity/v3/projects/9273d5abc65f4a45bb302211f5e69a28 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:19.472710 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad666dc-cf47-43fe-ac1f-a40a019a5833 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:19.473207 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad666dc-cf47-43fe-ac1f-a40a019a5833 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:19.473448 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad666dc-cf47-43fe-ac1f-a40a019a5833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:19.538832 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-9ad666dc-cf47-43fe-ac1f-a40a019a5833 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:19.551757 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-9ad666dc-cf47-43fe-ac1f-a40a019a5833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:19.770686 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 833/1660] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:42:19.781913 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970e151-734e-4ec8-b0c1-8409c3e39f67 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:19.782211 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970e151-734e-4ec8-b0c1-8409c3e39f67 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:19.782658 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970e151-734e-4ec8-b0c1-8409c3e39f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:19.838946 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-e970e151-734e-4ec8-b0c1-8409c3e39f67 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:19.841274 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-e970e151-734e-4ec8-b0c1-8409c3e39f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:19.854085 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 828/1661] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:19.860959 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad666dc-cf47-43fe-ac1f-a40a019a5833 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:19.916887 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad666dc-cf47-43fe-ac1f-a40a019a5833 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:19.925588 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75f86ef-bee1-4a07-9d47-a51ccc5621a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:19.925904 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75f86ef-bee1-4a07-9d47-a51ccc5621a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:19.930782 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75f86ef-bee1-4a07-9d47-a51ccc5621a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:19.931452 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75f86ef-bee1-4a07-9d47-a51ccc5621a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:19.932549 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75f86ef-bee1-4a07-9d47-a51ccc5621a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:42:19.932549 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75f86ef-bee1-4a07-9d47-a51ccc5621a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:19.935238 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75f86ef-bee1-4a07-9d47-a51ccc5621a9 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:19.988632 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e970e151-734e-4ec8-b0c1-8409c3e39f67 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:20.005365 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 834/1662] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 14:42:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:42:20.006955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e970e151-734e-4ec8-b0c1-8409c3e39f67 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:20.072233 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a684d463-40d0-4f3d-b7e4-8b9b4c91ffd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:20.076386 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684d463-40d0-4f3d-b7e4-8b9b4c91ffd4 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:20.076936 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684d463-40d0-4f3d-b7e4-8b9b4c91ffd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:20.078473 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684d463-40d0-4f3d-b7e4-8b9b4c91ffd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:20.080841 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a684d463-40d0-4f3d-b7e4-8b9b4c91ffd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:20.088118 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a684d463-40d0-4f3d-b7e4-8b9b4c91ffd4 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:20.181189 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 829/1663] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:42:19 2022] POST /identity/v3/projects => generated 391 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:20.188126 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75f86ef-bee1-4a07-9d47-a51ccc5621a9 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:20.204754 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75f86ef-bee1-4a07-9d47-a51ccc5621a9 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:20.388363 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5cd7c-0bbc-4f13-8bf4-a2b4620b3cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:20.388493 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5cd7c-0bbc-4f13-8bf4-a2b4620b3cf8 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:20.388768 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5cd7c-0bbc-4f13-8bf4-a2b4620b3cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:20.388962 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5cd7c-0bbc-4f13-8bf4-a2b4620b3cf8 admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:20.390759 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe5cd7c-0bbc-4f13-8bf4-a2b4620b3cf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:20.394329 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe5cd7c-0bbc-4f13-8bf4-a2b4620b3cf8 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:20.457220 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a684d463-40d0-4f3d-b7e4-8b9b4c91ffd4 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:20.457573 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 835/1664] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:42:20 2022] POST /identity/v3/users => generated 296 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:20.475638 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a684d463-40d0-4f3d-b7e4-8b9b4c91ffd4 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:20.491934 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c37f0-8c4e-4436-ba64-8f243ce3cc4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:20.492814 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c37f0-8c4e-4436-ba64-8f243ce3cc4b admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:20.493286 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c37f0-8c4e-4436-ba64-8f243ce3cc4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:20.493757 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c37f0-8c4e-4436-ba64-8f243ce3cc4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:20.494640 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122c37f0-8c4e-4436-ba64-8f243ce3cc4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:20.496888 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122c37f0-8c4e-4436-ba64-8f243ce3cc4b admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:20.527002 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 830/1665] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:20 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:20.536442 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5cd7c-0bbc-4f13-8bf4-a2b4620b3cf8 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:20.576035 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5cd7c-0bbc-4f13-8bf4-a2b4620b3cf8 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:20.593615 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6010569-8068-4425-936c-bfc68652aca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:20.594322 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6010569-8068-4425-936c-bfc68652aca1 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:20.594660 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6010569-8068-4425-936c-bfc68652aca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:20.594763 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6010569-8068-4425-936c-bfc68652aca1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:20.600090 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6010569-8068-4425-936c-bfc68652aca1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:20.602293 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6010569-8068-4425-936c-bfc68652aca1 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:20.646861 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 836/1666] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:20 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:20.651086 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c37f0-8c4e-4436-ba64-8f243ce3cc4b admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:20.678403 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c37f0-8c4e-4436-ba64-8f243ce3cc4b admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:20.694984 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c58128-c051-4d7b-ae3b-7def69df0bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:20.696142 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c58128-c051-4d7b-ae3b-7def69df0bd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:20.699883 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c58128-c051-4d7b-ae3b-7def69df0bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:20.700271 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c58128-c051-4d7b-ae3b-7def69df0bd0 admin admin] PATH_INFO: `/v3/projects/6525d35f462b4bf0a68465227e8cdcf7/users/6b02809a18d0490b942657907386ed54/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:20.758390 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c58128-c051-4d7b-ae3b-7def69df0bd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6525d35f462b4bf0a68465227e8cdcf7, user_id=6b02809a18d0490b942657907386ed54, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:20.763654 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c58128-c051-4d7b-ae3b-7def69df0bd0 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:20.801977 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 831/1667] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:42:20 2022] PUT /identity/v3/projects/6525d35f462b4bf0a68465227e8cdcf7/users/6b02809a18d0490b942657907386ed54/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:20.814276 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed350b-9a54-4703-8c2c-ec5a04cebdc5 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:20.819264 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed350b-9a54-4703-8c2c-ec5a04cebdc5 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:20.819880 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed350b-9a54-4703-8c2c-ec5a04cebdc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:20.947955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c58128-c051-4d7b-ae3b-7def69df0bd0 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:20.950683 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-15c58128-c051-4d7b-ae3b-7def69df0bd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:20.965563 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-c6ed350b-9a54-4703-8c2c-ec5a04cebdc5 None None] MFA Rules not processed for user `6b02809a18d0490b942657907386ed54`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:20.968783 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-c6ed350b-9a54-4703-8c2c-ec5a04cebdc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:21.017111 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:21.199944 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c58128-c051-4d7b-ae3b-7def69df0bd0 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:21.235399 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3964e3ce-60a7-439f-8149-f953282d911f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:21.237409 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964e3ce-60a7-439f-8149-f953282d911f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:21.239425 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964e3ce-60a7-439f-8149-f953282d911f admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:21.239824 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964e3ce-60a7-439f-8149-f953282d911f admin admin] PATH_INFO: `/v3/projects/6e9991d4feb042f19745ffb1d9c8b0cf` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:21.247187 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3964e3ce-60a7-439f-8149-f953282d911f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e9991d4feb042f19745ffb1d9c8b0cf)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:21.250012 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3964e3ce-60a7-439f-8149-f953282d911f admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:21.255442 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 837/1668] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:20 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 442 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:21.261087 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed350b-9a54-4703-8c2c-ec5a04cebdc5 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:21.283064 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed350b-9a54-4703-8c2c-ec5a04cebdc5 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:21.309768 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c5d743-191c-4c95-b96f-3aee1bfb7684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:21.314377 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5d743-191c-4c95-b96f-3aee1bfb7684 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:21.314866 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5d743-191c-4c95-b96f-3aee1bfb7684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:21.315783 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5d743-191c-4c95-b96f-3aee1bfb7684 admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:21.319442 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c5d743-191c-4c95-b96f-3aee1bfb7684 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:21.321387 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:21.325210 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c5d743-191c-4c95-b96f-3aee1bfb7684 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:21.409145 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 838/1669] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:42:21 2022] POST /identity/v3/projects => generated 393 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:21.413199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c5d743-191c-4c95-b96f-3aee1bfb7684 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:21.435842 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-3964e3ce-60a7-439f-8149-f953282d911f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e9991d4feb042f19745ffb1d9c8b0cf', 'request_id': 'req-3964e3ce-60a7-439f-8149-f953282d911f'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:21.448037 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-3964e3ce-60a7-439f-8149-f953282d911f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e9991d4feb042f19745ffb1d9c8b0cf', 'request_id': 'req-3964e3ce-60a7-439f-8149-f953282d911f'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:21.471993 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-3964e3ce-60a7-439f-8149-f953282d911f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e9991d4feb042f19745ffb1d9c8b0cf', 'request_id': 'req-3964e3ce-60a7-439f-8149-f953282d911f'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:21.507395 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-3964e3ce-60a7-439f-8149-f953282d911f admin admin] The token cache is being invalidate because project 6e9991d4feb042f19745ffb1d9c8b0cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:42:21.507804 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-3964e3ce-60a7-439f-8149-f953282d911f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:21.511022 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-3964e3ce-60a7-439f-8149-f953282d911f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:21.518838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 832/1670] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:42:20 2022] DELETE /identity/v3/projects/6e9991d4feb042f19745ffb1d9c8b0cf => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:21.521461 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018cc4e-6b6f-4023-a43d-2d5e643ecbdd None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:21.525090 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018cc4e-6b6f-4023-a43d-2d5e643ecbdd None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:21.525501 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018cc4e-6b6f-4023-a43d-2d5e643ecbdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:21.564374 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c5d743-191c-4c95-b96f-3aee1bfb7684 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:21.570643 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-1018cc4e-6b6f-4023-a43d-2d5e643ecbdd None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:21.572045 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b840bd27-c709-481f-9b16-29e8ce61d880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:21.572839 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840bd27-c709-481f-9b16-29e8ce61d880 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:21.573007 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-1018cc4e-6b6f-4023-a43d-2d5e643ecbdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:21.573416 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840bd27-c709-481f-9b16-29e8ce61d880 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:21.574106 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840bd27-c709-481f-9b16-29e8ce61d880 admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:21.575746 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b840bd27-c709-481f-9b16-29e8ce61d880 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:21.579340 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b840bd27-c709-481f-9b16-29e8ce61d880 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:21.612798 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 833/1671] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:21.614332 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac6eb40-11a7-4109-a0c6-4c0c1aacfdcf None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:21.619588 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac6eb40-11a7-4109-a0c6-4c0c1aacfdcf None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:21.622887 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac6eb40-11a7-4109-a0c6-4c0c1aacfdcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:21.636448 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 839/1672] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:42:21 2022] POST /identity/v3/users => generated 303 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:42:21.638843 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b840bd27-c709-481f-9b16-29e8ce61d880 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:21.642774 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-b840bd27-c709-481f-9b16-29e8ce61d880 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:21.670120 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-9ac6eb40-11a7-4109-a0c6-4c0c1aacfdcf None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:21.672384 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-9ac6eb40-11a7-4109-a0c6-4c0c1aacfdcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:21.690924 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b840bd27-c709-481f-9b16-29e8ce61d880 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:21.697177 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 834/1673] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:21.697411 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c898373-02e6-45a1-add7-2b0332ae777f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:21.698259 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c898373-02e6-45a1-add7-2b0332ae777f admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:21.698672 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c898373-02e6-45a1-add7-2b0332ae777f admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:21.699073 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c898373-02e6-45a1-add7-2b0332ae777f admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:21.700204 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c898373-02e6-45a1-add7-2b0332ae777f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:21.701390 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac6eb40-11a7-4109-a0c6-4c0c1aacfdcf None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:21.703672 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c898373-02e6-45a1-add7-2b0332ae777f admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:21.730156 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 840/1674] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:21 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:21.732638 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c898373-02e6-45a1-add7-2b0332ae777f admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:21.733390 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac6eb40-11a7-4109-a0c6-4c0c1aacfdcf None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:21.750971 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc409e0c-06ea-45e7-90f9-a292d6fcb3d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:21.751777 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc409e0c-06ea-45e7-90f9-a292d6fcb3d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:21.752589 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc409e0c-06ea-45e7-90f9-a292d6fcb3d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:21.752589 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc409e0c-06ea-45e7-90f9-a292d6fcb3d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:21.753887 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc409e0c-06ea-45e7-90f9-a292d6fcb3d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:42:21.754000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c898373-02e6-45a1-add7-2b0332ae777f admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:21.754387 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc409e0c-06ea-45e7-90f9-a292d6fcb3d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:21.765541 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc409e0c-06ea-45e7-90f9-a292d6fcb3d2 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:21.768984 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3068e25-0e90-4a8b-b364-588671e5336e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:21.772332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3068e25-0e90-4a8b-b364-588671e5336e admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:21.772332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3068e25-0e90-4a8b-b364-588671e5336e admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:21.775857 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3068e25-0e90-4a8b-b364-588671e5336e admin admin] PATH_INFO: `/v3/projects/c3ebd9afbec843a1a618cd3296665e3e/users/4d9843f47c5640f5ab7e0ab271f1662c/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:21.811753 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 835/1675] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 14:42:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:42:21.813584 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc409e0c-06ea-45e7-90f9-a292d6fcb3d2 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:21.830081 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3068e25-0e90-4a8b-b364-588671e5336e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3ebd9afbec843a1a618cd3296665e3e, user_id=4d9843f47c5640f5ab7e0ab271f1662c, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:21.844274 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc409e0c-06ea-45e7-90f9-a292d6fcb3d2 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:21.850362 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3068e25-0e90-4a8b-b364-588671e5336e admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:21.858083 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d411625-ea04-44f1-b73b-60caa93beb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:21.858655 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d411625-ea04-44f1-b73b-60caa93beb61 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:21.859603 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d411625-ea04-44f1-b73b-60caa93beb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:21.859975 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d411625-ea04-44f1-b73b-60caa93beb61 admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:21.860739 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d411625-ea04-44f1-b73b-60caa93beb61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:21.862922 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d411625-ea04-44f1-b73b-60caa93beb61 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:21.903841 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 841/1676] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:42:21 2022] PUT /identity/v3/projects/c3ebd9afbec843a1a618cd3296665e3e/users/4d9843f47c5640f5ab7e0ab271f1662c/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:21.910083 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3068e25-0e90-4a8b-b364-588671e5336e admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:21.920971 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 836/1677] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:42:21 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:21.926199 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d411625-ea04-44f1-b73b-60caa93beb61 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:22.066901 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:22.091301 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d411625-ea04-44f1-b73b-60caa93beb61 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:22.108299 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3068e25-0e90-4a8b-b364-588671e5336e admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:22.110157 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c8d653-b568-49d7-9fff-f2d7f7eff308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:22.111004 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c8d653-b568-49d7-9fff-f2d7f7eff308 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:22.111507 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c8d653-b568-49d7-9fff-f2d7f7eff308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:22.112237 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c8d653-b568-49d7-9fff-f2d7f7eff308 admin admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:22.113059 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c8d653-b568-49d7-9fff-f2d7f7eff308 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:22.115300 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c8d653-b568-49d7-9fff-f2d7f7eff308 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:22.115728 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c192b6c-6e62-4e5a-9f1d-93531da0f07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:22.116431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c192b6c-6e62-4e5a-9f1d-93531da0f07b admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:22.116836 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c192b6c-6e62-4e5a-9f1d-93531da0f07b admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:22.117589 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c192b6c-6e62-4e5a-9f1d-93531da0f07b admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:22.118667 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c192b6c-6e62-4e5a-9f1d-93531da0f07b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:22.120464 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c192b6c-6e62-4e5a-9f1d-93531da0f07b admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:22.134919 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 842/1678] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:21 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:22.143913 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c192b6c-6e62-4e5a-9f1d-93531da0f07b admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:22.173650 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 837/1679] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:42:21 2022] POST /identity/v3/users => generated 293 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:22.174854 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c8d653-b568-49d7-9fff-f2d7f7eff308 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:22.178730 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c192b6c-6e62-4e5a-9f1d-93531da0f07b admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:22.188834 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6c9937-ef29-4684-b6fe-43c9468addd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:22.192053 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c9937-ef29-4684-b6fe-43c9468addd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:22.192788 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c9937-ef29-4684-b6fe-43c9468addd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:22.195030 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c9937-ef29-4684-b6fe-43c9468addd0 admin admin] PATH_INFO: `/v3/projects/c3ebd9afbec843a1a618cd3296665e3e/users/4d9843f47c5640f5ab7e0ab271f1662c/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:22.196067 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c8d653-b568-49d7-9fff-f2d7f7eff308 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:22.203330 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6c9937-ef29-4684-b6fe-43c9468addd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3ebd9afbec843a1a618cd3296665e3e, user_id=4d9843f47c5640f5ab7e0ab271f1662c, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:22.208491 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6c9937-ef29-4684-b6fe-43c9468addd0 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:22.226160 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d559eaee-11cc-43cc-bbd1-71ab2f76b178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:22.227416 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559eaee-11cc-43cc-bbd1-71ab2f76b178 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:22.227924 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559eaee-11cc-43cc-bbd1-71ab2f76b178 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:22.228342 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559eaee-11cc-43cc-bbd1-71ab2f76b178 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:22.229143 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d559eaee-11cc-43cc-bbd1-71ab2f76b178 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:22.230910 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d559eaee-11cc-43cc-bbd1-71ab2f76b178 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:22.237173 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 843/1680] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:42:22 2022] PUT /identity/v3/projects/c3ebd9afbec843a1a618cd3296665e3e/users/4d9843f47c5640f5ab7e0ab271f1662c/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:22.254487 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d60057f-08e8-4b29-ad0b-7ecd2dcf14f3 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:22.255682 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 838/1681] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:22 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:22.256767 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d559eaee-11cc-43cc-bbd1-71ab2f76b178 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:22.259770 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d60057f-08e8-4b29-ad0b-7ecd2dcf14f3 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:22.260078 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d60057f-08e8-4b29-ad0b-7ecd2dcf14f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:22.282512 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d559eaee-11cc-43cc-bbd1-71ab2f76b178 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:22.348205 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-7d60057f-08e8-4b29-ad0b-7ecd2dcf14f3 None None] MFA Rules not processed for user `4d9843f47c5640f5ab7e0ab271f1662c`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:22.352737 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-7d60057f-08e8-4b29-ad0b-7ecd2dcf14f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:22.385359 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:22.468969 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ffc6a7-729c-4574-b91d-3ae78f02ab65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:22.471798 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ffc6a7-729c-4574-b91d-3ae78f02ab65 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:22.472877 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ffc6a7-729c-4574-b91d-3ae78f02ab65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:22.473263 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ffc6a7-729c-4574-b91d-3ae78f02ab65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:22.476027 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ffc6a7-729c-4574-b91d-3ae78f02ab65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:22.478154 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ffc6a7-729c-4574-b91d-3ae78f02ab65 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:22.502696 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 839/1682] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:22 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:22.506225 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ffc6a7-729c-4574-b91d-3ae78f02ab65 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:22.527970 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ffc6a7-729c-4574-b91d-3ae78f02ab65 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:22.541420 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac95b6-e907-4905-948b-ad4b29a3deb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:22.542484 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac95b6-e907-4905-948b-ad4b29a3deb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:22.542974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac95b6-e907-4905-948b-ad4b29a3deb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:22.543504 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac95b6-e907-4905-948b-ad4b29a3deb8 admin admin] PATH_INFO: `/v3/projects/f9defcdb1e814fed80f62b83435ea777/users/c85150eca9c84ffe82a69e73fc32bd36/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:22.560235 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 844/1683] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:22 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:22.574017 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d60057f-08e8-4b29-ad0b-7ecd2dcf14f3 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:22.576177 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-7d60057f-08e8-4b29-ad0b-7ecd2dcf14f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:22.592722 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ac95b6-e907-4905-948b-ad4b29a3deb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9defcdb1e814fed80f62b83435ea777, user_id=c85150eca9c84ffe82a69e73fc32bd36, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:22.596808 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ac95b6-e907-4905-948b-ad4b29a3deb8 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:22.623680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 840/1684] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:42:22 2022] PUT /identity/v3/projects/f9defcdb1e814fed80f62b83435ea777/users/c85150eca9c84ffe82a69e73fc32bd36/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:22.635784 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c460f8-4647-4739-9f31-b3dc90edbb1e None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:22.636453 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c460f8-4647-4739-9f31-b3dc90edbb1e None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:22.637498 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c460f8-4647-4739-9f31-b3dc90edbb1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:22.722642 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d60057f-08e8-4b29-ad0b-7ecd2dcf14f3 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:22.737304 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-87c460f8-4647-4739-9f31-b3dc90edbb1e None None] MFA Rules not processed for user `c85150eca9c84ffe82a69e73fc32bd36`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:22.738885 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-87c460f8-4647-4739-9f31-b3dc90edbb1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:22.864092 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ffe90a-1144-4d3d-9df5-e9694d291264 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:22.865241 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ffe90a-1144-4d3d-9df5-e9694d291264 service cinder] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:22.865600 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ffe90a-1144-4d3d-9df5-e9694d291264 service cinder] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:22.865956 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ffe90a-1144-4d3d-9df5-e9694d291264 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:22.903822 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:22.970420 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ffe90a-1144-4d3d-9df5-e9694d291264 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:22.972731 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ffe90a-1144-4d3d-9df5-e9694d291264 service cinder] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:22.976980 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 841/1685] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:42:22.982334 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c460f8-4647-4739-9f31-b3dc90edbb1e None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:22.987027 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 845/1686] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:22 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:42:23.114022 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ffe90a-1144-4d3d-9df5-e9694d291264 service cinder] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:23.114812 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-a5ffe90a-1144-4d3d-9df5-e9694d291264 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:23.135732 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ffe90a-1144-4d3d-9df5-e9694d291264 service cinder] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:23.146699 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf6a3be-f94c-4537-9892-f1bfb8699cb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:23.147549 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6a3be-f94c-4537-9892-f1bfb8699cb7 service cinder] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:23.149617 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6a3be-f94c-4537-9892-f1bfb8699cb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:23.150129 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6a3be-f94c-4537-9892-f1bfb8699cb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:23.166275 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-5bf6a3be-f94c-4537-9892-f1bfb8699cb7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:23.198496 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c460f8-4647-4739-9f31-b3dc90edbb1e None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:23.232572 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc9c1ed-db3b-4877-8a5e-bf5349454384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:23.233231 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc9c1ed-db3b-4877-8a5e-bf5349454384 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:23.233480 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc9c1ed-db3b-4877-8a5e-bf5349454384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:23.233819 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc9c1ed-db3b-4877-8a5e-bf5349454384 admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:23.234515 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc9c1ed-db3b-4877-8a5e-bf5349454384 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:23.236525 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc9c1ed-db3b-4877-8a5e-bf5349454384 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:23.301828 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 842/1687] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:42:22 2022] POST /identity/v3/projects => generated 387 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:23.302787 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc9c1ed-db3b-4877-8a5e-bf5349454384 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:23.346105 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:23.412086 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf6a3be-f94c-4537-9892-f1bfb8699cb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:23.414381 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf6a3be-f94c-4537-9892-f1bfb8699cb7 service cinder] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:23.454667 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc9c1ed-db3b-4877-8a5e-bf5349454384 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:23.456659 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 846/1688] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:23 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:23.466529 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92ff22f-06c9-4186-8aba-450a6582d177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:23.467149 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ff22f-06c9-4186-8aba-450a6582d177 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:23.467456 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ff22f-06c9-4186-8aba-450a6582d177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:23.467743 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ff22f-06c9-4186-8aba-450a6582d177 admin admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:23.468970 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92ff22f-06c9-4186-8aba-450a6582d177 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:23.472993 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92ff22f-06c9-4186-8aba-450a6582d177 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:23.517657 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 843/1689] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:42:23 2022] POST /identity/v3/users => generated 300 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:23.521546 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf6a3be-f94c-4537-9892-f1bfb8699cb7 service cinder] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:23.536330 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf6a3be-f94c-4537-9892-f1bfb8699cb7 service cinder] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:23.543093 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b608e1-525a-4675-b812-183a05d6ed58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:23.544216 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b608e1-525a-4675-b812-183a05d6ed58 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:23.544718 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b608e1-525a-4675-b812-183a05d6ed58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:23.545204 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b608e1-525a-4675-b812-183a05d6ed58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:23.546141 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b608e1-525a-4675-b812-183a05d6ed58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:23.550961 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b608e1-525a-4675-b812-183a05d6ed58 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:23.566076 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 847/1690] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:23 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:23.569888 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92ff22f-06c9-4186-8aba-450a6582d177 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:23.582884 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92ff22f-06c9-4186-8aba-450a6582d177 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:23.610691 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55614b0d-fd30-4a9f-9965-b8e1647d8969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:23.611687 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55614b0d-fd30-4a9f-9965-b8e1647d8969 admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:23.612227 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55614b0d-fd30-4a9f-9965-b8e1647d8969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:23.612747 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55614b0d-fd30-4a9f-9965-b8e1647d8969 admin admin] PATH_INFO: `/v3/projects/b76daa9fc9474da195371038ba9ac526/users/3d1d5167e6f848ccac4067b313c03544/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:23.651908 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55614b0d-fd30-4a9f-9965-b8e1647d8969 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b76daa9fc9474da195371038ba9ac526, user_id=3d1d5167e6f848ccac4067b313c03544, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:23.657499 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55614b0d-fd30-4a9f-9965-b8e1647d8969 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:23.689860 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b608e1-525a-4675-b812-183a05d6ed58 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:23.690701 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 844/1691] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:42:23 2022] PUT /identity/v3/projects/b76daa9fc9474da195371038ba9ac526/users/3d1d5167e6f848ccac4067b313c03544/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:23.713370 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b608e1-525a-4675-b812-183a05d6ed58 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:23.837945 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8cb169-f839-48c8-b8d2-65ba9e8483a0 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:23.838232 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8cb169-f839-48c8-b8d2-65ba9e8483a0 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:23.838508 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8cb169-f839-48c8-b8d2-65ba9e8483a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:23.863165 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e080a877-0b7b-49d2-ac5d-9a405f8e9cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:23.866118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e080a877-0b7b-49d2-ac5d-9a405f8e9cab admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:23.866425 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e080a877-0b7b-49d2-ac5d-9a405f8e9cab admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:23.868128 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e080a877-0b7b-49d2-ac5d-9a405f8e9cab admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:23.869586 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e080a877-0b7b-49d2-ac5d-9a405f8e9cab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:23.876514 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e080a877-0b7b-49d2-ac5d-9a405f8e9cab admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:23.898218 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-5f8cb169-f839-48c8-b8d2-65ba9e8483a0 None None] MFA Rules not processed for user `aafc618b52ae496299fdf53d2cac771a`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:23.899585 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-5f8cb169-f839-48c8-b8d2-65ba9e8483a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:23.904782 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 848/1692] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:23 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:42:23.909560 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e080a877-0b7b-49d2-ac5d-9a405f8e9cab admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:23.933651 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e080a877-0b7b-49d2-ac5d-9a405f8e9cab admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:23.947421 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33032900-8227-45d3-991a-8c39b5fe3176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:23.948375 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33032900-8227-45d3-991a-8c39b5fe3176 admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:23.948375 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33032900-8227-45d3-991a-8c39b5fe3176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:23.948665 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33032900-8227-45d3-991a-8c39b5fe3176 admin admin] PATH_INFO: `/v3/projects/b76daa9fc9474da195371038ba9ac526/users/3d1d5167e6f848ccac4067b313c03544/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:23.956928 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33032900-8227-45d3-991a-8c39b5fe3176 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b76daa9fc9474da195371038ba9ac526, user_id=3d1d5167e6f848ccac4067b313c03544, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:23.963933 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33032900-8227-45d3-991a-8c39b5fe3176 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:24.014541 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 849/1693] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:42:23 2022] PUT /identity/v3/projects/b76daa9fc9474da195371038ba9ac526/users/3d1d5167e6f848ccac4067b313c03544/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:42:24.025523 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f3ba36-f5e6-4809-8e78-0a137f20dd3a None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:24.025883 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f3ba36-f5e6-4809-8e78-0a137f20dd3a None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:24.026251 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f3ba36-f5e6-4809-8e78-0a137f20dd3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:24.112284 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 845/1694] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:23 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:42:24.115345 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8cb169-f839-48c8-b8d2-65ba9e8483a0 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:24.118324 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-5f8cb169-f839-48c8-b8d2-65ba9e8483a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:24.119348 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-91f3ba36-f5e6-4809-8e78-0a137f20dd3a None None] MFA Rules not processed for user `3d1d5167e6f848ccac4067b313c03544`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:24.123369 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-91f3ba36-f5e6-4809-8e78-0a137f20dd3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:24.168198 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8cb169-f839-48c8-b8d2-65ba9e8483a0 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:24.180090 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:24.267891 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822fddfa-17bd-4c30-b0f7-6f2866bc71e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:24.269184 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822fddfa-17bd-4c30-b0f7-6f2866bc71e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:24.269670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822fddfa-17bd-4c30-b0f7-6f2866bc71e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:24.270200 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822fddfa-17bd-4c30-b0f7-6f2866bc71e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:24.271004 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822fddfa-17bd-4c30-b0f7-6f2866bc71e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:24.274474 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822fddfa-17bd-4c30-b0f7-6f2866bc71e3 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:24.308438 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:24.330693 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 846/1695] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:42:24 2022] POST /identity/v3/projects => generated 405 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:24.336495 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822fddfa-17bd-4c30-b0f7-6f2866bc71e3 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:24.344966 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 850/1696] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:24 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:24.357318 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f3ba36-f5e6-4809-8e78-0a137f20dd3a None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:24.430863 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822fddfa-17bd-4c30-b0f7-6f2866bc71e3 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:24.438314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb230a26-35e7-4a57-8020-ed2024f82dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:24.438947 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb230a26-35e7-4a57-8020-ed2024f82dc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:24.439249 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb230a26-35e7-4a57-8020-ed2024f82dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:24.439529 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb230a26-35e7-4a57-8020-ed2024f82dc7 admin admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:24.440920 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb230a26-35e7-4a57-8020-ed2024f82dc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:24.443168 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb230a26-35e7-4a57-8020-ed2024f82dc7 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:24.460753 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f3ba36-f5e6-4809-8e78-0a137f20dd3a None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:24.488255 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 847/1697] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:42:24 2022] POST /identity/v3/users => generated 309 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:42:24.489667 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb230a26-35e7-4a57-8020-ed2024f82dc7 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:24.512640 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb230a26-35e7-4a57-8020-ed2024f82dc7 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:24.541472 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d412ac-f8d4-4ab4-855c-1602978bd944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:24.542425 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d412ac-f8d4-4ab4-855c-1602978bd944 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:24.543765 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d412ac-f8d4-4ab4-855c-1602978bd944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:24.544111 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d412ac-f8d4-4ab4-855c-1602978bd944 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:24.547835 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d412ac-f8d4-4ab4-855c-1602978bd944 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:24.553801 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d412ac-f8d4-4ab4-855c-1602978bd944 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:24.571962 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11254a43-988b-4024-a260-f750e48bb39e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:24.574960 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11254a43-988b-4024-a260-f750e48bb39e service cinder] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:24.575273 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11254a43-988b-4024-a260-f750e48bb39e service cinder] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:24.575551 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11254a43-988b-4024-a260-f750e48bb39e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:24.589213 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 848/1698] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:24 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:42:24.597689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d412ac-f8d4-4ab4-855c-1602978bd944 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:24.636090 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d412ac-f8d4-4ab4-855c-1602978bd944 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:24.659232 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babef9b4-5d88-41d5-9c1a-e8711ef96f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:24.663326 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babef9b4-5d88-41d5-9c1a-e8711ef96f9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:24.663460 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babef9b4-5d88-41d5-9c1a-e8711ef96f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:24.663642 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babef9b4-5d88-41d5-9c1a-e8711ef96f9c admin admin] PATH_INFO: `/v3/projects/81bd4c6161a841efa7813d962322ff5d/users/c06c0ad183804721a047e34e37a115de/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:24.723052 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babef9b4-5d88-41d5-9c1a-e8711ef96f9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=81bd4c6161a841efa7813d962322ff5d, user_id=c06c0ad183804721a047e34e37a115de, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:24.738872 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babef9b4-5d88-41d5-9c1a-e8711ef96f9c admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:24.782865 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 849/1699] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:42:24 2022] PUT /identity/v3/projects/81bd4c6161a841efa7813d962322ff5d/users/c06c0ad183804721a047e34e37a115de/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:24.784616 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babef9b4-5d88-41d5-9c1a-e8711ef96f9c admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:24.800273 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11254a43-988b-4024-a260-f750e48bb39e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:24.807052 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11254a43-988b-4024-a260-f750e48bb39e service cinder] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:24.836971 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 851/1700] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:24 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:42:24.911047 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11254a43-988b-4024-a260-f750e48bb39e service cinder] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:24.939459 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11254a43-988b-4024-a260-f750e48bb39e service cinder] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:24.952466 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab0677-ca61-4f2c-b07f-1691ec22f429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:24.955805 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab0677-ca61-4f2c-b07f-1691ec22f429 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:24.955977 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab0677-ca61-4f2c-b07f-1691ec22f429 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:24.956228 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab0677-ca61-4f2c-b07f-1691ec22f429 admin admin] PATH_INFO: `/v3/users/c85150eca9c84ffe82a69e73fc32bd36` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:24.959927 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab0677-ca61-4f2c-b07f-1691ec22f429 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c85150eca9c84ffe82a69e73fc32bd36)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:24.962623 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babef9b4-5d88-41d5-9c1a-e8711ef96f9c admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:24.966472 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab0677-ca61-4f2c-b07f-1691ec22f429 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:24.975824 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd2c9e-2687-440c-8c02-f290f949ad49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:24.976583 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd2c9e-2687-440c-8c02-f290f949ad49 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:24.976979 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd2c9e-2687-440c-8c02-f290f949ad49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:24.977358 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd2c9e-2687-440c-8c02-f290f949ad49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:24.978112 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd2c9e-2687-440c-8c02-f290f949ad49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:24.980880 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd2c9e-2687-440c-8c02-f290f949ad49 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:25.003535 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:25.009071 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 850/1701] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:24 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:25.011800 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd2c9e-2687-440c-8c02-f290f949ad49 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:25.040915 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd2c9e-2687-440c-8c02-f290f949ad49 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:25.052302 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d91c42e-d8dd-48b2-837e-f37bbc7252c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:25.053351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d91c42e-d8dd-48b2-837e-f37bbc7252c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:25.053778 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d91c42e-d8dd-48b2-837e-f37bbc7252c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:25.058234 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d91c42e-d8dd-48b2-837e-f37bbc7252c0 admin admin] PATH_INFO: `/v3/projects/81bd4c6161a841efa7813d962322ff5d/users/c06c0ad183804721a047e34e37a115de/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:25.062181 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d91c42e-d8dd-48b2-837e-f37bbc7252c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81bd4c6161a841efa7813d962322ff5d, user_id=c06c0ad183804721a047e34e37a115de, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:25.074950 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-96ab0677-ca61-4f2c-b07f-1691ec22f429 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c85150eca9c84ffe82a69e73fc32bd36'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:25.076108 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-96ab0677-ca61-4f2c-b07f-1691ec22f429 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c85150eca9c84ffe82a69e73fc32bd36'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:25.076530 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d91c42e-d8dd-48b2-837e-f37bbc7252c0 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:25.090892 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-96ab0677-ca61-4f2c-b07f-1691ec22f429 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c85150eca9c84ffe82a69e73fc32bd36'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:25.108141 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 851/1702] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:42:25 2022] PUT /identity/v3/projects/81bd4c6161a841efa7813d962322ff5d/users/c06c0ad183804721a047e34e37a115de/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:25.108381 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-96ab0677-ca61-4f2c-b07f-1691ec22f429 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c85150eca9c84ffe82a69e73fc32bd36'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:25.109762 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-96ab0677-ca61-4f2c-b07f-1691ec22f429 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c85150eca9c84ffe82a69e73fc32bd36'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:25.139579 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 852/1703] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:42:24 2022] DELETE /identity/v3/users/c85150eca9c84ffe82a69e73fc32bd36 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:25.146001 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf43a4-4f19-40e9-8eac-0b1661364ec4 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:25.146620 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf43a4-4f19-40e9-8eac-0b1661364ec4 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:25.147176 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf43a4-4f19-40e9-8eac-0b1661364ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:25.193635 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-3dbf43a4-4f19-40e9-8eac-0b1661364ec4 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:25.196159 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-3dbf43a4-4f19-40e9-8eac-0b1661364ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:25.313687 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:25.407860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 852/1704] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:25.483166 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab0677-ca61-4f2c-b07f-1691ec22f429 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:25.486252 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-96ab0677-ca61-4f2c-b07f-1691ec22f429 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:25.695736 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab0677-ca61-4f2c-b07f-1691ec22f429 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:25.706148 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506769dd-5c7f-470b-8bbb-6675e05873a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:25.706829 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506769dd-5c7f-470b-8bbb-6675e05873a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:25.707120 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506769dd-5c7f-470b-8bbb-6675e05873a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:25.707426 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506769dd-5c7f-470b-8bbb-6675e05873a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:25.721522 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506769dd-5c7f-470b-8bbb-6675e05873a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:25.723394 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506769dd-5c7f-470b-8bbb-6675e05873a1 service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:25.743437 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 853/1705] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:42:25.914151 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbf43a4-4f19-40e9-8eac-0b1661364ec4 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:25.915696 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-3dbf43a4-4f19-40e9-8eac-0b1661364ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:25.957325 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbf43a4-4f19-40e9-8eac-0b1661364ec4 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:25.972236 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215ccc2-69ea-4915-8cbc-c20c92760be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:25.973503 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215ccc2-69ea-4915-8cbc-c20c92760be1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:25.974198 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215ccc2-69ea-4915-8cbc-c20c92760be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:25.974478 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215ccc2-69ea-4915-8cbc-c20c92760be1 admin admin] PATH_INFO: `/v3/users/dad848db58cb45a8b0a9b0f74b9632ed` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:25.979731 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9215ccc2-69ea-4915-8cbc-c20c92760be1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dad848db58cb45a8b0a9b0f74b9632ed)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:25.985659 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9215ccc2-69ea-4915-8cbc-c20c92760be1 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:26.120566 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-9215ccc2-69ea-4915-8cbc-c20c92760be1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dad848db58cb45a8b0a9b0f74b9632ed'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:26.122089 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-9215ccc2-69ea-4915-8cbc-c20c92760be1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dad848db58cb45a8b0a9b0f74b9632ed'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:26.156944 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-9215ccc2-69ea-4915-8cbc-c20c92760be1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dad848db58cb45a8b0a9b0f74b9632ed'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:26.194060 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-9215ccc2-69ea-4915-8cbc-c20c92760be1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dad848db58cb45a8b0a9b0f74b9632ed'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:26.230668 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-9215ccc2-69ea-4915-8cbc-c20c92760be1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dad848db58cb45a8b0a9b0f74b9632ed'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:26.246766 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 853/1706] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:42:25 2022] DELETE /identity/v3/users/dad848db58cb45a8b0a9b0f74b9632ed => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:26.251516 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1653faf9-26b1-4a2f-a343-f2958b7c11e8 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:26.251960 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1653faf9-26b1-4a2f-a343-f2958b7c11e8 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:26.252064 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1653faf9-26b1-4a2f-a343-f2958b7c11e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:26.296459 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-1653faf9-26b1-4a2f-a343-f2958b7c11e8 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:26.298858 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-1653faf9-26b1-4a2f-a343-f2958b7c11e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:26.515992 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 854/1707] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:26.579547 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215ccc2-69ea-4915-8cbc-c20c92760be1 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:26.581081 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-9215ccc2-69ea-4915-8cbc-c20c92760be1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:26.630026 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5cd0c4-d291-439e-a7ab-4eee985865ed None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:26.630846 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5cd0c4-d291-439e-a7ab-4eee985865ed None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:26.631166 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5cd0c4-d291-439e-a7ab-4eee985865ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:26.754187 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-8e5cd0c4-d291-439e-a7ab-4eee985865ed None None] MFA Rules not processed for user `0642fa92a86c4e01a42bf087cd36cace`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:26.773537 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-8e5cd0c4-d291-439e-a7ab-4eee985865ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:26.838726 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215ccc2-69ea-4915-8cbc-c20c92760be1 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:26.860610 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:26.865603 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4252c36d-5121-44f0-a8e8-21eccbed8957 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:26.870275 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252c36d-5121-44f0-a8e8-21eccbed8957 service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:26.871920 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252c36d-5121-44f0-a8e8-21eccbed8957 service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:26.872583 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252c36d-5121-44f0-a8e8-21eccbed8957 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:26.911719 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4252c36d-5121-44f0-a8e8-21eccbed8957 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:26.914331 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4252c36d-5121-44f0-a8e8-21eccbed8957 service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:26.931971 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 854/1708] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:27.016693 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 855/1709] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:42:27.030104 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7385bea1-5c0a-4344-a961-2d40352c99e1 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:27.030469 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7385bea1-5c0a-4344-a961-2d40352c99e1 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:27.030770 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7385bea1-5c0a-4344-a961-2d40352c99e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:27.092869 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-7385bea1-5c0a-4344-a961-2d40352c99e1 None None] MFA Rules not processed for user `0642fa92a86c4e01a42bf087cd36cace`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:27.100865 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-7385bea1-5c0a-4344-a961-2d40352c99e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:27.142085 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 855/1710] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:27.155126 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5cd0c4-d291-439e-a7ab-4eee985865ed None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:27.158008 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-8e5cd0c4-d291-439e-a7ab-4eee985865ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:27.377698 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5cd0c4-d291-439e-a7ab-4eee985865ed None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:27.394630 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74bc8dd-3bb6-42e0-95f1-cc1a9a86200f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:27.395388 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74bc8dd-3bb6-42e0-95f1-cc1a9a86200f service nova] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:27.395937 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74bc8dd-3bb6-42e0-95f1-cc1a9a86200f service nova] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:27.396216 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74bc8dd-3bb6-42e0-95f1-cc1a9a86200f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:27.432831 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74bc8dd-3bb6-42e0-95f1-cc1a9a86200f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:27.434771 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74bc8dd-3bb6-42e0-95f1-cc1a9a86200f service nova] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:27.457507 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 856/1711] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:27 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:28.094457 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7385bea1-5c0a-4344-a961-2d40352c99e1 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:28.097120 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-7385bea1-5c0a-4344-a961-2d40352c99e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:28.126486 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7385bea1-5c0a-4344-a961-2d40352c99e1 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:28.161130 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c95f2-e6f3-476d-a650-c317596f0abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:28.161802 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c95f2-e6f3-476d-a650-c317596f0abc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:28.162079 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c95f2-e6f3-476d-a650-c317596f0abc admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:28.162351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c95f2-e6f3-476d-a650-c317596f0abc admin admin] PATH_INFO: `/v3/projects/f9defcdb1e814fed80f62b83435ea777` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:28.176494 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c95f2-e6f3-476d-a650-c317596f0abc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9defcdb1e814fed80f62b83435ea777)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:28.178892 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c95f2-e6f3-476d-a650-c317596f0abc admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:28.305018 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:28.323026 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-de8c95f2-e6f3-476d-a650-c317596f0abc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9defcdb1e814fed80f62b83435ea777', 'request_id': 'req-de8c95f2-e6f3-476d-a650-c317596f0abc'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:28.333298 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-de8c95f2-e6f3-476d-a650-c317596f0abc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9defcdb1e814fed80f62b83435ea777', 'request_id': 'req-de8c95f2-e6f3-476d-a650-c317596f0abc'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:28.341955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-de8c95f2-e6f3-476d-a650-c317596f0abc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9defcdb1e814fed80f62b83435ea777', 'request_id': 'req-de8c95f2-e6f3-476d-a650-c317596f0abc'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:28.358183 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-de8c95f2-e6f3-476d-a650-c317596f0abc admin admin] The token cache is being invalidate because project f9defcdb1e814fed80f62b83435ea777 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:42:28.359521 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-de8c95f2-e6f3-476d-a650-c317596f0abc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:28.360431 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-de8c95f2-e6f3-476d-a650-c317596f0abc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:28.362677 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 856/1712] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:42:28 2022] DELETE /identity/v3/projects/f9defcdb1e814fed80f62b83435ea777 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:28.366403 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74bc8dd-3bb6-42e0-95f1-cc1a9a86200f service nova] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:28.383078 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74bc8dd-3bb6-42e0-95f1-cc1a9a86200f service nova] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:28.530025 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9d67ad-2bef-4043-82ff-c9aa4ee776d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:28.530897 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d67ad-2bef-4043-82ff-c9aa4ee776d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:28.534737 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d67ad-2bef-4043-82ff-c9aa4ee776d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:28.535497 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d67ad-2bef-4043-82ff-c9aa4ee776d5 admin admin] PATH_INFO: `/v3/users/3d1d5167e6f848ccac4067b313c03544` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:28.537411 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9d67ad-2bef-4043-82ff-c9aa4ee776d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d1d5167e6f848ccac4067b313c03544)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:28.540061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9d67ad-2bef-4043-82ff-c9aa4ee776d5 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:28.689627 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-dc9d67ad-2bef-4043-82ff-c9aa4ee776d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d1d5167e6f848ccac4067b313c03544'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:28.692475 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-dc9d67ad-2bef-4043-82ff-c9aa4ee776d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d1d5167e6f848ccac4067b313c03544'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:28.704269 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:28.713389 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-dc9d67ad-2bef-4043-82ff-c9aa4ee776d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d1d5167e6f848ccac4067b313c03544'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:28.729989 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-dc9d67ad-2bef-4043-82ff-c9aa4ee776d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d1d5167e6f848ccac4067b313c03544'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:28.731772 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-dc9d67ad-2bef-4043-82ff-c9aa4ee776d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d1d5167e6f848ccac4067b313c03544'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:28.785368 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 857/1713] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:42:28 2022] DELETE /identity/v3/users/3d1d5167e6f848ccac4067b313c03544 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:28.856330 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c95f2-e6f3-476d-a650-c317596f0abc admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:28.857946 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-de8c95f2-e6f3-476d-a650-c317596f0abc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:28.886398 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c95f2-e6f3-476d-a650-c317596f0abc admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:29.084144 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32571f1b-353c-4488-b2ce-82df9507a79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:29.084822 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32571f1b-353c-4488-b2ce-82df9507a79f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:29.085100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32571f1b-353c-4488-b2ce-82df9507a79f admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:29.085368 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32571f1b-353c-4488-b2ce-82df9507a79f admin admin] PATH_INFO: `/v3/users/6b02809a18d0490b942657907386ed54` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:29.086557 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32571f1b-353c-4488-b2ce-82df9507a79f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b02809a18d0490b942657907386ed54)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:29.088570 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32571f1b-353c-4488-b2ce-82df9507a79f admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:29.204480 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-32571f1b-353c-4488-b2ce-82df9507a79f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b02809a18d0490b942657907386ed54'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:29.205878 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-32571f1b-353c-4488-b2ce-82df9507a79f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b02809a18d0490b942657907386ed54'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:29.228286 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-32571f1b-353c-4488-b2ce-82df9507a79f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b02809a18d0490b942657907386ed54'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:29.252287 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-32571f1b-353c-4488-b2ce-82df9507a79f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b02809a18d0490b942657907386ed54'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:29.264212 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-32571f1b-353c-4488-b2ce-82df9507a79f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b02809a18d0490b942657907386ed54'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:29.271575 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 857/1714] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:42:28 2022] DELETE /identity/v3/users/6b02809a18d0490b942657907386ed54 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:29.279043 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c035e-c855-45b3-ac18-232aa825b0e5 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:29.280728 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c035e-c855-45b3-ac18-232aa825b0e5 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:29.282908 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c035e-c855-45b3-ac18-232aa825b0e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:29.342738 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-859c035e-c855-45b3-ac18-232aa825b0e5 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:29.345708 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5024b-f016-4cf1-abb6-5d80052546dc None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:29.346220 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5024b-f016-4cf1-abb6-5d80052546dc None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:29.346802 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5024b-f016-4cf1-abb6-5d80052546dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:29.348926 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-859c035e-c855-45b3-ac18-232aa825b0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:29.421730 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-80f5024b-f016-4cf1-abb6-5d80052546dc None None] MFA Rules not processed for user `35f28006916040c3b988de6c15093166`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:29.423101 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-80f5024b-f016-4cf1-abb6-5d80052546dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:29.515389 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:29.587596 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 858/1715] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:29.589937 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc599ec9-2071-45df-8dc3-14d8f5998c55 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:29.590383 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc599ec9-2071-45df-8dc3-14d8f5998c55 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:29.590811 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc599ec9-2071-45df-8dc3-14d8f5998c55 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:29.593637 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 859/1716] 10.222.0.30 () {54 vars in 912 bytes} [Tue Aug 30 14:42:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:42:29.599363 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc599ec9-2071-45df-8dc3-14d8f5998c55 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:29.603329 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 858/1717] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:29 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:29.603826 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-cc599ec9-2071-45df-8dc3-14d8f5998c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:29.607321 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6e7b64-8166-43c7-ab32-20b669afa12b None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:29.607726 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6e7b64-8166-43c7-ab32-20b669afa12b None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:29.608017 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6e7b64-8166-43c7-ab32-20b669afa12b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:29.734775 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-1a6e7b64-8166-43c7-ab32-20b669afa12b None None] MFA Rules not processed for user `9e14e86e085d4987aece69fe64867ce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:29.736334 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-1a6e7b64-8166-43c7-ab32-20b669afa12b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:29.773032 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc599ec9-2071-45df-8dc3-14d8f5998c55 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:29.781310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1826867-33e7-4216-9e27-889bb19d6e30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:29.783715 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1826867-33e7-4216-9e27-889bb19d6e30 service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:29.785700 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1826867-33e7-4216-9e27-889bb19d6e30 service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:29.785993 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1826867-33e7-4216-9e27-889bb19d6e30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:29.834498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1826867-33e7-4216-9e27-889bb19d6e30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:29.836759 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1826867-33e7-4216-9e27-889bb19d6e30 service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:29.847922 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 860/1718] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:29.849048 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c4e2fb-0463-4423-8a1e-66c2bebc68fa None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:29.849381 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c4e2fb-0463-4423-8a1e-66c2bebc68fa None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:29.849606 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c4e2fb-0463-4423-8a1e-66c2bebc68fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:29.887764 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-85c4e2fb-0463-4423-8a1e-66c2bebc68fa None None] MFA Rules not processed for user `35f28006916040c3b988de6c15093166`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:29.888135 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 859/1719] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Aug 30 14:42:29 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:29.889093 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-85c4e2fb-0463-4423-8a1e-66c2bebc68fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:29.889258 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6e7b64-8166-43c7-ab32-20b669afa12b None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:29.893110 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-1a6e7b64-8166-43c7-ab32-20b669afa12b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:29.898362 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 861/1720] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:29 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:29.900318 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9375bd06-f942-4cb4-8dd2-6c3e96d0bde9 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:29.900613 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9375bd06-f942-4cb4-8dd2-6c3e96d0bde9 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:29.900955 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9375bd06-f942-4cb4-8dd2-6c3e96d0bde9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:29.932631 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6e7b64-8166-43c7-ab32-20b669afa12b None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:29.948214 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9addeac7-d8bd-4dae-86a2-d35b1f3f7da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:29.948880 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9addeac7-d8bd-4dae-86a2-d35b1f3f7da8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:29.949157 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9addeac7-d8bd-4dae-86a2-d35b1f3f7da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:29.949426 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9addeac7-d8bd-4dae-86a2-d35b1f3f7da8 admin admin] PATH_INFO: `/v3/projects/9db62f1e868a420680e55a9c81d79817` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:29.951287 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9addeac7-d8bd-4dae-86a2-d35b1f3f7da8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9db62f1e868a420680e55a9c81d79817)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:29.956323 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9addeac7-d8bd-4dae-86a2-d35b1f3f7da8 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:29.958061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-9375bd06-f942-4cb4-8dd2-6c3e96d0bde9 None None] MFA Rules not processed for user `c06c0ad183804721a047e34e37a115de`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:29.959398 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-9375bd06-f942-4cb4-8dd2-6c3e96d0bde9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:30.078008 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-9addeac7-d8bd-4dae-86a2-d35b1f3f7da8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9db62f1e868a420680e55a9c81d79817', 'request_id': 'req-9addeac7-d8bd-4dae-86a2-d35b1f3f7da8'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:30.092590 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-9addeac7-d8bd-4dae-86a2-d35b1f3f7da8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9db62f1e868a420680e55a9c81d79817', 'request_id': 'req-9addeac7-d8bd-4dae-86a2-d35b1f3f7da8'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:30.094224 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:30.103694 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-9addeac7-d8bd-4dae-86a2-d35b1f3f7da8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9db62f1e868a420680e55a9c81d79817', 'request_id': 'req-9addeac7-d8bd-4dae-86a2-d35b1f3f7da8'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:30.134607 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-9addeac7-d8bd-4dae-86a2-d35b1f3f7da8 admin admin] The token cache is being invalidate because project 9db62f1e868a420680e55a9c81d79817 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:42:30.134909 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-9addeac7-d8bd-4dae-86a2-d35b1f3f7da8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:30.141955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-9addeac7-d8bd-4dae-86a2-d35b1f3f7da8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:30.149473 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 860/1721] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:42:29 2022] DELETE /identity/v3/projects/9db62f1e868a420680e55a9c81d79817 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:30.150507 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9addeac7-d8bd-4dae-86a2-d35b1f3f7da8 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:30.154842 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 862/1722] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:29 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:30.156707 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9375bd06-f942-4cb4-8dd2-6c3e96d0bde9 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:30.158406 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-9375bd06-f942-4cb4-8dd2-6c3e96d0bde9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:30.161132 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-9addeac7-d8bd-4dae-86a2-d35b1f3f7da8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:30.187436 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9375bd06-f942-4cb4-8dd2-6c3e96d0bde9 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:30.289517 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e47fb24-f3bd-4362-99c4-85bf61eb802b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:30.291198 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47fb24-f3bd-4362-99c4-85bf61eb802b service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:30.291516 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47fb24-f3bd-4362-99c4-85bf61eb802b service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:30.291790 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47fb24-f3bd-4362-99c4-85bf61eb802b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:30.294236 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-4e47fb24-f3bd-4362-99c4-85bf61eb802b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:30.295808 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:30.343841 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e47fb24-f3bd-4362-99c4-85bf61eb802b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:30.347040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e47fb24-f3bd-4362-99c4-85bf61eb802b service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:30.352934 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9addeac7-d8bd-4dae-86a2-d35b1f3f7da8 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:30.370459 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e59f22-1aa4-4f7a-802f-131e377e58e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:30.371090 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e59f22-1aa4-4f7a-802f-131e377e58e5 service nova] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:30.371395 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e59f22-1aa4-4f7a-802f-131e377e58e5 service nova] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:30.371667 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e59f22-1aa4-4f7a-802f-131e377e58e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:30.378235 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-89e59f22-1aa4-4f7a-802f-131e377e58e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:30.382285 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 863/1723] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:30.385673 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e38b44-55be-4fff-89f7-211eda6e82aa None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:30.386128 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e38b44-55be-4fff-89f7-211eda6e82aa None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:30.386563 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e38b44-55be-4fff-89f7-211eda6e82aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:30.422767 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-a0e38b44-55be-4fff-89f7-211eda6e82aa None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:30.438355 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-a0e38b44-55be-4fff-89f7-211eda6e82aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:30.529598 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e59f22-1aa4-4f7a-802f-131e377e58e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:30.534820 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e59f22-1aa4-4f7a-802f-131e377e58e5 service nova] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:30.561779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 861/1724] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:30 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:30.564460 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e59f22-1aa4-4f7a-802f-131e377e58e5 service nova] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:30.571009 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-89e59f22-1aa4-4f7a-802f-131e377e58e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:30.650356 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 864/1725] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:30.660814 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5099d-3427-44c0-b88e-9a7a20ad7212 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:30.661246 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5099d-3427-44c0-b88e-9a7a20ad7212 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:30.661579 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5099d-3427-44c0-b88e-9a7a20ad7212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:30.745787 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-40a5099d-3427-44c0-b88e-9a7a20ad7212 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:30.748733 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-40a5099d-3427-44c0-b88e-9a7a20ad7212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:30.762511 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 865/1726] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:30.776551 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a5099d-3427-44c0-b88e-9a7a20ad7212 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:30.798967 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e59f22-1aa4-4f7a-802f-131e377e58e5 service nova] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:30.815557 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a5099d-3427-44c0-b88e-9a7a20ad7212 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:30.831987 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922890ba-83e7-451c-83eb-55f23d605d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:30.832655 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922890ba-83e7-451c-83eb-55f23d605d11 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:30.832940 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922890ba-83e7-451c-83eb-55f23d605d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:30.833218 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922890ba-83e7-451c-83eb-55f23d605d11 admin admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:30.834475 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922890ba-83e7-451c-83eb-55f23d605d11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:42:30.834475 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922890ba-83e7-451c-83eb-55f23d605d11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:30.835858 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922890ba-83e7-451c-83eb-55f23d605d11 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:30.837194 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e48a2-4a44-41fe-b591-c3a8230c69e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:30.838148 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e48a2-4a44-41fe-b591-c3a8230c69e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:30.838425 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e48a2-4a44-41fe-b591-c3a8230c69e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:30.838698 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e48a2-4a44-41fe-b591-c3a8230c69e6 admin admin] PATH_INFO: `/v3/projects/6525d35f462b4bf0a68465227e8cdcf7` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:30.844844 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7e48a2-4a44-41fe-b591-c3a8230c69e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6525d35f462b4bf0a68465227e8cdcf7)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:30.846796 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7e48a2-4a44-41fe-b591-c3a8230c69e6 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:30.862410 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 866/1727] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 14:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:42:30.869919 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922890ba-83e7-451c-83eb-55f23d605d11 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:30.888259 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922890ba-83e7-451c-83eb-55f23d605d11 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:30.902119 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84837f2-d6f3-40ef-92ed-f8aea7373e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:30.902808 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84837f2-d6f3-40ef-92ed-f8aea7373e61 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:30.903094 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84837f2-d6f3-40ef-92ed-f8aea7373e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:30.903396 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84837f2-d6f3-40ef-92ed-f8aea7373e61 admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:30.904048 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84837f2-d6f3-40ef-92ed-f8aea7373e61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:30.906069 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84837f2-d6f3-40ef-92ed-f8aea7373e61 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:30.980623 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 867/1728] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:42:30 2022] POST /identity/v3/projects => generated 379 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:30.985268 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84837f2-d6f3-40ef-92ed-f8aea7373e61 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:30.993507 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-6b7e48a2-4a44-41fe-b591-c3a8230c69e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6525d35f462b4bf0a68465227e8cdcf7', 'request_id': 'req-6b7e48a2-4a44-41fe-b591-c3a8230c69e6'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:31.022445 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-6b7e48a2-4a44-41fe-b591-c3a8230c69e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6525d35f462b4bf0a68465227e8cdcf7', 'request_id': 'req-6b7e48a2-4a44-41fe-b591-c3a8230c69e6'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:31.044358 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-6b7e48a2-4a44-41fe-b591-c3a8230c69e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6525d35f462b4bf0a68465227e8cdcf7', 'request_id': 'req-6b7e48a2-4a44-41fe-b591-c3a8230c69e6'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:31.074794 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-6b7e48a2-4a44-41fe-b591-c3a8230c69e6 admin admin] The token cache is being invalidate because project 6525d35f462b4bf0a68465227e8cdcf7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:42:31.075087 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-6b7e48a2-4a44-41fe-b591-c3a8230c69e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:31.077099 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-6b7e48a2-4a44-41fe-b591-c3a8230c69e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:31.079126 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 862/1729] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:42:30 2022] DELETE /identity/v3/projects/6525d35f462b4bf0a68465227e8cdcf7 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:31.083104 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e48a2-4a44-41fe-b591-c3a8230c69e6 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:31.085583 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-6b7e48a2-4a44-41fe-b591-c3a8230c69e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:31.107973 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:31.145119 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84837f2-d6f3-40ef-92ed-f8aea7373e61 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:31.156648 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:31.160478 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea250858-d7d9-43f8-ab10-d8df973710c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:31.161113 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea250858-d7d9-43f8-ab10-d8df973710c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:31.161390 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea250858-d7d9-43f8-ab10-d8df973710c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:31.161725 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea250858-d7d9-43f8-ab10-d8df973710c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:31.162946 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea250858-d7d9-43f8-ab10-d8df973710c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:31.164547 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea250858-d7d9-43f8-ab10-d8df973710c2 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:31.213118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 868/1730] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:42:30 2022] POST /identity/v3/users => generated 290 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:42:31.217763 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea250858-d7d9-43f8-ab10-d8df973710c2 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:31.222077 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-ea250858-d7d9-43f8-ab10-d8df973710c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:31.244800 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea250858-d7d9-43f8-ab10-d8df973710c2 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:31.251862 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e48a2-4a44-41fe-b591-c3a8230c69e6 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:31.258643 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c600c-fbdf-4309-b70c-1ac813668ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:31.259539 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c600c-fbdf-4309-b70c-1ac813668ab5 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:31.259972 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c600c-fbdf-4309-b70c-1ac813668ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:31.260399 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c600c-fbdf-4309-b70c-1ac813668ab5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:31.261234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348c600c-fbdf-4309-b70c-1ac813668ab5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:31.263364 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348c600c-fbdf-4309-b70c-1ac813668ab5 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:31.266452 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad92048-5c62-41af-ad88-417e76dfc07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:31.267363 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad92048-5c62-41af-ad88-417e76dfc07e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:31.267836 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad92048-5c62-41af-ad88-417e76dfc07e admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:31.268287 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad92048-5c62-41af-ad88-417e76dfc07e admin admin] PATH_INFO: `/v3/users/4d9843f47c5640f5ab7e0ab271f1662c` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:31.269711 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad92048-5c62-41af-ad88-417e76dfc07e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d9843f47c5640f5ab7e0ab271f1662c)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:31.272893 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad92048-5c62-41af-ad88-417e76dfc07e admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:31.286965 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 869/1731] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:42:31.298336 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c600c-fbdf-4309-b70c-1ac813668ab5 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:31.318768 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c600c-fbdf-4309-b70c-1ac813668ab5 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:31.330243 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bacb11-1d10-417a-afe2-aba069195171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:31.331040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bacb11-1d10-417a-afe2-aba069195171 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:31.331425 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bacb11-1d10-417a-afe2-aba069195171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:31.333832 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bacb11-1d10-417a-afe2-aba069195171 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:31.335665 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bacb11-1d10-417a-afe2-aba069195171 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:31.337803 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bacb11-1d10-417a-afe2-aba069195171 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:31.378941 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 870/1732] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:31.382762 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bacb11-1d10-417a-afe2-aba069195171 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:31.385177 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-4ad92048-5c62-41af-ad88-417e76dfc07e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d9843f47c5640f5ab7e0ab271f1662c'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:31.385995 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-4ad92048-5c62-41af-ad88-417e76dfc07e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d9843f47c5640f5ab7e0ab271f1662c'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:31.388724 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-46bacb11-1d10-417a-afe2-aba069195171 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:31.413933 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bacb11-1d10-417a-afe2-aba069195171 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:31.415834 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-4ad92048-5c62-41af-ad88-417e76dfc07e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d9843f47c5640f5ab7e0ab271f1662c'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:31.422660 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533a2c3-de79-4fc3-b3dd-3978c0030ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:31.423815 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533a2c3-de79-4fc3-b3dd-3978c0030ae0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:31.424296 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533a2c3-de79-4fc3-b3dd-3978c0030ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:31.425071 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533a2c3-de79-4fc3-b3dd-3978c0030ae0 admin admin] PATH_INFO: `/v3/projects/b76daa9fc9474da195371038ba9ac526` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:31.427012 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2533a2c3-de79-4fc3-b3dd-3978c0030ae0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b76daa9fc9474da195371038ba9ac526)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:31.427826 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-4ad92048-5c62-41af-ad88-417e76dfc07e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d9843f47c5640f5ab7e0ab271f1662c'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:31.429824 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2533a2c3-de79-4fc3-b3dd-3978c0030ae0 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:31.447505 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-4ad92048-5c62-41af-ad88-417e76dfc07e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d9843f47c5640f5ab7e0ab271f1662c'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:31.468959 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 863/1733] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:42:31 2022] DELETE /identity/v3/users/4d9843f47c5640f5ab7e0ab271f1662c => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:31.470282 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad92048-5c62-41af-ad88-417e76dfc07e admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:31.472383 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-4ad92048-5c62-41af-ad88-417e76dfc07e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:31.523381 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-2533a2c3-de79-4fc3-b3dd-3978c0030ae0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b76daa9fc9474da195371038ba9ac526', 'request_id': 'req-2533a2c3-de79-4fc3-b3dd-3978c0030ae0'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:31.530924 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-2533a2c3-de79-4fc3-b3dd-3978c0030ae0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b76daa9fc9474da195371038ba9ac526', 'request_id': 'req-2533a2c3-de79-4fc3-b3dd-3978c0030ae0'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:31.540326 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-2533a2c3-de79-4fc3-b3dd-3978c0030ae0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b76daa9fc9474da195371038ba9ac526', 'request_id': 'req-2533a2c3-de79-4fc3-b3dd-3978c0030ae0'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:31.578766 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-2533a2c3-de79-4fc3-b3dd-3978c0030ae0 admin admin] The token cache is being invalidate because project b76daa9fc9474da195371038ba9ac526 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:42:31.579159 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-2533a2c3-de79-4fc3-b3dd-3978c0030ae0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:31.580323 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-2533a2c3-de79-4fc3-b3dd-3978c0030ae0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:31.585215 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 871/1734] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:42:31 2022] DELETE /identity/v3/projects/b76daa9fc9474da195371038ba9ac526 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:31.604350 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a912fe8-0c17-4660-832f-cf23aa9da9b0 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:31.605766 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a912fe8-0c17-4660-832f-cf23aa9da9b0 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:31.606240 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a912fe8-0c17-4660-832f-cf23aa9da9b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:31.705709 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-7a912fe8-0c17-4660-832f-cf23aa9da9b0 None None] MFA Rules not processed for user `ca9bafc5ff4e43719f37e62cad946cf7`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:31.708546 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-7a912fe8-0c17-4660-832f-cf23aa9da9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:31.714160 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad92048-5c62-41af-ad88-417e76dfc07e admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:31.820662 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:31.849973 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 872/1735] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:31.852744 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145bba8b-c71a-4e2e-a839-b617af82f373 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:31.853086 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145bba8b-c71a-4e2e-a839-b617af82f373 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:31.853369 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145bba8b-c71a-4e2e-a839-b617af82f373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:31.859609 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30f745-5527-4468-bb06-332d1e4a2426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:31.860900 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30f745-5527-4468-bb06-332d1e4a2426 admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:31.861599 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30f745-5527-4468-bb06-332d1e4a2426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:31.862123 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30f745-5527-4468-bb06-332d1e4a2426 admin admin] PATH_INFO: `/v3/projects/f07273ec04bb4262a5667aa7f3af3a5b/users/128cc020d9454b798e9a2ccfe1bff86c/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:31.887869 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-145bba8b-c71a-4e2e-a839-b617af82f373 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:31.892456 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-145bba8b-c71a-4e2e-a839-b617af82f373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:31.897145 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b30f745-5527-4468-bb06-332d1e4a2426 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f07273ec04bb4262a5667aa7f3af3a5b, user_id=128cc020d9454b798e9a2ccfe1bff86c, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:31.900050 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b30f745-5527-4468-bb06-332d1e4a2426 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:31.919268 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 873/1736] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:31.924029 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2316fb-fe8b-4607-8973-19e79c997a15 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:31.924305 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2316fb-fe8b-4607-8973-19e79c997a15 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:31.924555 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2316fb-fe8b-4607-8973-19e79c997a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:31.934486 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 864/1737] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:42:31 2022] PUT /identity/v3/projects/f07273ec04bb4262a5667aa7f3af3a5b/users/128cc020d9454b798e9a2ccfe1bff86c/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:31.947092 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226eaa2a-84da-411e-acd1-bad420a3de94 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:31.947539 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226eaa2a-84da-411e-acd1-bad420a3de94 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:31.947812 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226eaa2a-84da-411e-acd1-bad420a3de94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:31.987266 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-3f2316fb-fe8b-4607-8973-19e79c997a15 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:31.988868 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-3f2316fb-fe8b-4607-8973-19e79c997a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:32.021009 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-226eaa2a-84da-411e-acd1-bad420a3de94 None None] MFA Rules not processed for user `128cc020d9454b798e9a2ccfe1bff86c`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:32.025876 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-226eaa2a-84da-411e-acd1-bad420a3de94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:32.044978 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:32.151667 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 874/1738] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:42:32.152556 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2316fb-fe8b-4607-8973-19e79c997a15 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:32.157562 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-3f2316fb-fe8b-4607-8973-19e79c997a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:32.166144 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 865/1739] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:42:32.168162 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226eaa2a-84da-411e-acd1-bad420a3de94 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:32.174942 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-226eaa2a-84da-411e-acd1-bad420a3de94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:32.274336 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2316fb-fe8b-4607-8973-19e79c997a15 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:32.287914 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9ef240-89ca-485c-b6b9-72a15fbdd674 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:32.288846 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ef240-89ca-485c-b6b9-72a15fbdd674 service nova] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:32.289318 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ef240-89ca-485c-b6b9-72a15fbdd674 service nova] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:32.289760 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ef240-89ca-485c-b6b9-72a15fbdd674 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:32.346303 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226eaa2a-84da-411e-acd1-bad420a3de94 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:32.363732 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2351b-a3b5-430f-b6a3-91f5ee95e534 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcf302a170934c25b39699088dac326a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:32.364363 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2351b-a3b5-430f-b6a3-91f5ee95e534 service placement] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:32.364678 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2351b-a3b5-430f-b6a3-91f5ee95e534 service placement] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:32.365141 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2351b-a3b5-430f-b6a3-91f5ee95e534 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:32.370320 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-d3e2351b-a3b5-430f-b6a3-91f5ee95e534 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:32.395023 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e2351b-a3b5-430f-b6a3-91f5ee95e534 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:32.399797 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e2351b-a3b5-430f-b6a3-91f5ee95e534 service placement] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:32.423167 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 866/1740] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:32.425951 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2351b-a3b5-430f-b6a3-91f5ee95e534 service placement] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:32.428499 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9ef240-89ca-485c-b6b9-72a15fbdd674 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:32.430260 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9ef240-89ca-485c-b6b9-72a15fbdd674 service nova] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:32.467440 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 875/1741] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:32.469724 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2351b-a3b5-430f-b6a3-91f5ee95e534 service placement] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:32.470791 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9ef240-89ca-485c-b6b9-72a15fbdd674 service nova] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:32.472825 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-2f9ef240-89ca-485c-b6b9-72a15fbdd674 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:32.481120 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351eaf9-80ac-4abc-8316-01104fd8b6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:32.481943 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351eaf9-80ac-4abc-8316-01104fd8b6bc admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:32.486262 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351eaf9-80ac-4abc-8316-01104fd8b6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:32.487033 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351eaf9-80ac-4abc-8316-01104fd8b6bc admin admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:32.489012 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8351eaf9-80ac-4abc-8316-01104fd8b6bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:42:32.492000 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8351eaf9-80ac-4abc-8316-01104fd8b6bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:32.493877 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8351eaf9-80ac-4abc-8316-01104fd8b6bc admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:32.510718 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9ef240-89ca-485c-b6b9-72a15fbdd674 service nova] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:32.523807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 867/1742] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 14:42:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:42:32.526166 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de80fa5-557a-4fe3-81c0-515709e31bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:32.527143 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de80fa5-557a-4fe3-81c0-515709e31bc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:32.527693 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de80fa5-557a-4fe3-81c0-515709e31bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:32.528121 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de80fa5-557a-4fe3-81c0-515709e31bc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:32.528908 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de80fa5-557a-4fe3-81c0-515709e31bc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:32.529954 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351eaf9-80ac-4abc-8316-01104fd8b6bc admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:32.533143 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de80fa5-557a-4fe3-81c0-515709e31bc0 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:32.537288 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-8351eaf9-80ac-4abc-8316-01104fd8b6bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:32.565890 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351eaf9-80ac-4abc-8316-01104fd8b6bc admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:32.576872 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0d965c-2945-4893-aa7f-b75d5f0fb9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:32.584124 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d965c-2945-4893-aa7f-b75d5f0fb9aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:32.584410 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d965c-2945-4893-aa7f-b75d5f0fb9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:32.584686 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d965c-2945-4893-aa7f-b75d5f0fb9aa admin admin] PATH_INFO: `/v3/projects/c3ebd9afbec843a1a618cd3296665e3e` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:32.587220 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0d965c-2945-4893-aa7f-b75d5f0fb9aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3ebd9afbec843a1a618cd3296665e3e)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:32.589154 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0d965c-2945-4893-aa7f-b75d5f0fb9aa admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:32.595936 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 876/1743] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:42:32 2022] POST /identity/v3/projects => generated 381 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:32.595936 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ca147-de58-4b1e-bb7b-a5a9bf92dd3a None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:32.595936 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ca147-de58-4b1e-bb7b-a5a9bf92dd3a None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:32.595936 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ca147-de58-4b1e-bb7b-a5a9bf92dd3a None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:32.601411 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 877/1744] 10.222.0.30 () {54 vars in 921 bytes} [Tue Aug 30 14:42:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:42:32.602596 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ca147-de58-4b1e-bb7b-a5a9bf92dd3a None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:32.678431 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:32.728022 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-fd0d965c-2945-4893-aa7f-b75d5f0fb9aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3ebd9afbec843a1a618cd3296665e3e', 'request_id': 'req-fd0d965c-2945-4893-aa7f-b75d5f0fb9aa'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:32.748182 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-fd0d965c-2945-4893-aa7f-b75d5f0fb9aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3ebd9afbec843a1a618cd3296665e3e', 'request_id': 'req-fd0d965c-2945-4893-aa7f-b75d5f0fb9aa'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:32.754231 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:32.776442 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-fd0d965c-2945-4893-aa7f-b75d5f0fb9aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3ebd9afbec843a1a618cd3296665e3e', 'request_id': 'req-fd0d965c-2945-4893-aa7f-b75d5f0fb9aa'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:32.817841 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-fd0d965c-2945-4893-aa7f-b75d5f0fb9aa admin admin] The token cache is being invalidate because project c3ebd9afbec843a1a618cd3296665e3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:42:32.818268 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-fd0d965c-2945-4893-aa7f-b75d5f0fb9aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:32.824104 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-fd0d965c-2945-4893-aa7f-b75d5f0fb9aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:32.825123 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ca147-de58-4b1e-bb7b-a5a9bf92dd3a None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:32.829210 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 868/1745] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:42:32 2022] DELETE /identity/v3/projects/c3ebd9afbec843a1a618cd3296665e3e => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:32.830534 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f6f6b-106b-4a95-95c7-025fb047d6ed None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:32.830818 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f6f6b-106b-4a95-95c7-025fb047d6ed None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:32.831097 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f6f6b-106b-4a95-95c7-025fb047d6ed None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:32.852104 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 869/1746] 10.222.0.30 () {54 vars in 912 bytes} [Tue Aug 30 14:42:32 2022] GET /identity => generated 270 bytes in 22 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:42:32.853115 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1f6f6b-106b-4a95-95c7-025fb047d6ed None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:32.866012 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-5d1f6f6b-106b-4a95-95c7-025fb047d6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:33.084421 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa86bd7-9a7d-43cd-beb7-f399f9a9425c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:33.085061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa86bd7-9a7d-43cd-beb7-f399f9a9425c admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:33.085338 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa86bd7-9a7d-43cd-beb7-f399f9a9425c admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:33.085680 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa86bd7-9a7d-43cd-beb7-f399f9a9425c admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:33.086240 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa86bd7-9a7d-43cd-beb7-f399f9a9425c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:33.091098 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa86bd7-9a7d-43cd-beb7-f399f9a9425c admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:33.164065 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1f6f6b-106b-4a95-95c7-025fb047d6ed None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:33.193221 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becbc4c7-3a1a-49b8-8a99-c56191b63922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:33.195936 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becbc4c7-3a1a-49b8-8a99-c56191b63922 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:33.196371 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becbc4c7-3a1a-49b8-8a99-c56191b63922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:33.196791 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becbc4c7-3a1a-49b8-8a99-c56191b63922 admin admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:33.197756 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becbc4c7-3a1a-49b8-8a99-c56191b63922 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:33.205424 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becbc4c7-3a1a-49b8-8a99-c56191b63922 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:33.215359 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 878/1747] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:42:32 2022] POST /identity/v3/projects => generated 375 bytes in 613 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:33.216846 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3265a-6c0a-4de4-bda5-6a5ccb2316e4 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:33.220864 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3265a-6c0a-4de4-bda5-6a5ccb2316e4 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:33.221331 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3265a-6c0a-4de4-bda5-6a5ccb2316e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:33.268548 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 870/1748] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:42:32 2022] POST /identity/v3/users => generated 297 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:33.272066 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2a323d-b6a7-4cf0-88a9-1786235122c1 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:33.279778 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2a323d-b6a7-4cf0-88a9-1786235122c1 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:33.280044 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2a323d-b6a7-4cf0-88a9-1786235122c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:33.286522 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-e2f3265a-6c0a-4de4-bda5-6a5ccb2316e4 None None] MFA Rules not processed for user `dcf302a170934c25b39699088dac326a`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:33.287918 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-e2f3265a-6c0a-4de4-bda5-6a5ccb2316e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:33.324655 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-3b2a323d-b6a7-4cf0-88a9-1786235122c1 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:33.329123 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-3b2a323d-b6a7-4cf0-88a9-1786235122c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:33.477613 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 879/1749] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Aug 30 14:42:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:33.478906 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e88253-1090-4bf5-890c-3b10d41fc7c9 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:33.479192 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e88253-1090-4bf5-890c-3b10d41fc7c9 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:33.480616 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e88253-1090-4bf5-890c-3b10d41fc7c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:33.506465 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 871/1750] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:33.507491 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2a323d-b6a7-4cf0-88a9-1786235122c1 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:33.513209 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-3b2a323d-b6a7-4cf0-88a9-1786235122c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:33.538073 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-22e88253-1090-4bf5-890c-3b10d41fc7c9 None None] MFA Rules not processed for user `9e14e86e085d4987aece69fe64867ce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:33.539452 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-22e88253-1090-4bf5-890c-3b10d41fc7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:33.544970 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2a323d-b6a7-4cf0-88a9-1786235122c1 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:33.568828 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1b443e-7421-4334-82ef-3c8f75fd2ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:33.570190 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b443e-7421-4334-82ef-3c8f75fd2ee4 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:33.570190 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b443e-7421-4334-82ef-3c8f75fd2ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:33.570190 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b443e-7421-4334-82ef-3c8f75fd2ee4 admin admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:33.570616 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1b443e-7421-4334-82ef-3c8f75fd2ee4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:33.572643 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1b443e-7421-4334-82ef-3c8f75fd2ee4 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:33.630923 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 872/1751] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:42:33 2022] POST /identity/v3/users => generated 288 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:33.630923 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1b443e-7421-4334-82ef-3c8f75fd2ee4 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:33.653376 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1b443e-7421-4334-82ef-3c8f75fd2ee4 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:33.684100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c64ee2e-d930-476e-93c9-df7a76eac8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:33.684100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c64ee2e-d930-476e-93c9-df7a76eac8f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:33.684100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c64ee2e-d930-476e-93c9-df7a76eac8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:33.684100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c64ee2e-d930-476e-93c9-df7a76eac8f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:33.684559 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c64ee2e-d930-476e-93c9-df7a76eac8f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:33.684559 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c64ee2e-d930-476e-93c9-df7a76eac8f0 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:33.710958 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 873/1752] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:33 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:33.712321 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a41f25-979f-457f-9da7-e8a5c7bdd799 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:33.712611 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a41f25-979f-457f-9da7-e8a5c7bdd799 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:33.715517 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a41f25-979f-457f-9da7-e8a5c7bdd799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:33.717525 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 880/1753] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Aug 30 14:42:33 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:33.719523 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d35f83a-fc2e-4cba-a47d-e53291263dfa None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:33.719811 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d35f83a-fc2e-4cba-a47d-e53291263dfa None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:33.720121 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d35f83a-fc2e-4cba-a47d-e53291263dfa None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:33.722268 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 881/1754] 10.222.0.30 () {54 vars in 961 bytes} [Tue Aug 30 14:42:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:42:33.723145 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d35f83a-fc2e-4cba-a47d-e53291263dfa None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:33.756223 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d35f83a-fc2e-4cba-a47d-e53291263dfa None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:33.772421 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34e39a4-41ba-4650-b923-667eae4b399e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:33.774001 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-b8a41f25-979f-457f-9da7-e8a5c7bdd799 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:33.774091 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34e39a4-41ba-4650-b923-667eae4b399e admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:33.774357 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34e39a4-41ba-4650-b923-667eae4b399e admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:33.774635 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34e39a4-41ba-4650-b923-667eae4b399e admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:33.775046 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-b8a41f25-979f-457f-9da7-e8a5c7bdd799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:33.778452 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34e39a4-41ba-4650-b923-667eae4b399e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:33.781134 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34e39a4-41ba-4650-b923-667eae4b399e admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:33.801820 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 874/1755] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:33.808853 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a41f25-979f-457f-9da7-e8a5c7bdd799 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:33.815471 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 882/1756] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:33 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:33.816381 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb37ac0-8aff-47d5-ba99-3c6d94a1395c None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:33.818857 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb37ac0-8aff-47d5-ba99-3c6d94a1395c None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:33.819154 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb37ac0-8aff-47d5-ba99-3c6d94a1395c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:33.835921 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a41f25-979f-457f-9da7-e8a5c7bdd799 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:33.852154 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22728b8-2119-477f-b0f7-a3de677c522d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:33.853396 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22728b8-2119-477f-b0f7-a3de677c522d admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:33.853713 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22728b8-2119-477f-b0f7-a3de677c522d admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:33.853990 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22728b8-2119-477f-b0f7-a3de677c522d admin admin] PATH_INFO: `/v3/projects/3ddeeff8356d4ab08a6260ca7750343c/users/4246de3e7143447d934fb7dde53b3dcc/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:33.863577 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-6fb37ac0-8aff-47d5-ba99-3c6d94a1395c None None] MFA Rules not processed for user `dcf302a170934c25b39699088dac326a`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:33.865039 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-6fb37ac0-8aff-47d5-ba99-3c6d94a1395c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:33.885904 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=88,keystone:DELETE=5,keystone:INSERT=7 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:33.898662 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22728b8-2119-477f-b0f7-a3de677c522d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ddeeff8356d4ab08a6260ca7750343c, user_id=4246de3e7143447d934fb7dde53b3dcc, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:33.899584 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 883/1757] 10.222.0.30 () {58 vars in 1058 bytes} [Tue Aug 30 14:42:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:33.900474 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb37ac0-8aff-47d5-ba99-3c6d94a1395c None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:33.905863 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22728b8-2119-477f-b0f7-a3de677c522d admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:33.910611 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:33.926320 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb37ac0-8aff-47d5-ba99-3c6d94a1395c None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:33.933439 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8433cb80-330b-4885-8aa3-b970f38f11f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:33.934695 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8433cb80-330b-4885-8aa3-b970f38f11f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:33.935624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8433cb80-330b-4885-8aa3-b970f38f11f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:33.936173 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8433cb80-330b-4885-8aa3-b970f38f11f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:33.938059 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8433cb80-330b-4885-8aa3-b970f38f11f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:42:33.938592 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8433cb80-330b-4885-8aa3-b970f38f11f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:33.941108 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8433cb80-330b-4885-8aa3-b970f38f11f2 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:33.953092 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 875/1758] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:42:33 2022] PUT /identity/v3/projects/3ddeeff8356d4ab08a6260ca7750343c/users/4246de3e7143447d934fb7dde53b3dcc/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:33.956245 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22728b8-2119-477f-b0f7-a3de677c522d admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:33.975032 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 884/1759] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 14:42:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:42:33.979368 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8433cb80-330b-4885-8aa3-b970f38f11f2 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:33.986364 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-8433cb80-330b-4885-8aa3-b970f38f11f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:34.014029 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8433cb80-330b-4885-8aa3-b970f38f11f2 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:34.144782 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22728b8-2119-477f-b0f7-a3de677c522d admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:34.153821 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee58e6ba-7eb2-48b7-92a4-401d7e6091ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:34.154611 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee58e6ba-7eb2-48b7-92a4-401d7e6091ab service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:34.156407 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee58e6ba-7eb2-48b7-92a4-401d7e6091ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:34.158185 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee58e6ba-7eb2-48b7-92a4-401d7e6091ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:34.159383 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4bc730-aa3a-4f7a-acaa-e3e9366a7c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:34.161752 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4bc730-aa3a-4f7a-acaa-e3e9366a7c44 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:34.162104 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4bc730-aa3a-4f7a-acaa-e3e9366a7c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:34.162543 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4bc730-aa3a-4f7a-acaa-e3e9366a7c44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:34.163934 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4bc730-aa3a-4f7a-acaa-e3e9366a7c44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:34.165644 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4bc730-aa3a-4f7a-acaa-e3e9366a7c44 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:34.178635 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee58e6ba-7eb2-48b7-92a4-401d7e6091ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:34.184529 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee58e6ba-7eb2-48b7-92a4-401d7e6091ab service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:34.195001 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 876/1760] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:33 2022] GET /identity/v3/roles => generated 1420 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:34.196214 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4bc730-aa3a-4f7a-acaa-e3e9366a7c44 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:34.207056 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 885/1761] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:34.208971 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee58e6ba-7eb2-48b7-92a4-401d7e6091ab service neutron] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:34.240169 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee58e6ba-7eb2-48b7-92a4-401d7e6091ab service neutron] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:34.260624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58256b48-08f7-4fd5-a288-c156016af4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:34.261416 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58256b48-08f7-4fd5-a288-c156016af4a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:34.261700 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58256b48-08f7-4fd5-a288-c156016af4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:34.261971 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58256b48-08f7-4fd5-a288-c156016af4a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:34.263012 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58256b48-08f7-4fd5-a288-c156016af4a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:34.265062 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58256b48-08f7-4fd5-a288-c156016af4a4 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:34.304433 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 886/1762] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:34 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:34.305264 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58256b48-08f7-4fd5-a288-c156016af4a4 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:34.336259 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58256b48-08f7-4fd5-a288-c156016af4a4 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:34.355245 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5098cb3f-aeb4-41b5-a0f0-ccdc3f9e9f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:34.355925 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5098cb3f-aeb4-41b5-a0f0-ccdc3f9e9f09 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:34.356231 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5098cb3f-aeb4-41b5-a0f0-ccdc3f9e9f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:34.356506 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5098cb3f-aeb4-41b5-a0f0-ccdc3f9e9f09 admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:34.363258 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5098cb3f-aeb4-41b5-a0f0-ccdc3f9e9f09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:34.365267 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5098cb3f-aeb4-41b5-a0f0-ccdc3f9e9f09 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:34.375698 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4bc730-aa3a-4f7a-acaa-e3e9366a7c44 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:34.395737 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0cc272-5dc6-4586-a9d9-d422546cb378 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcf302a170934c25b39699088dac326a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:34.397466 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0cc272-5dc6-4586-a9d9-d422546cb378 service placement] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:34.398823 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0cc272-5dc6-4586-a9d9-d422546cb378 service placement] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:34.399107 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0cc272-5dc6-4586-a9d9-d422546cb378 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:34.421028 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0cc272-5dc6-4586-a9d9-d422546cb378 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:34.424748 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0cc272-5dc6-4586-a9d9-d422546cb378 service placement] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:34.449918 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 877/1763] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:34.453765 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0cc272-5dc6-4586-a9d9-d422546cb378 service placement] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:34.453864 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 887/1764] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:42:34 2022] POST /identity/v3/projects => generated 401 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:34.455356 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5098cb3f-aeb4-41b5-a0f0-ccdc3f9e9f09 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:34.477213 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0cc272-5dc6-4586-a9d9-d422546cb378 service placement] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:34.669595 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af726e11-d6f8-4f07-ac13-8c96932929b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:34.671856 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af726e11-d6f8-4f07-ac13-8c96932929b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:34.672371 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af726e11-d6f8-4f07-ac13-8c96932929b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:34.672847 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af726e11-d6f8-4f07-ac13-8c96932929b5 admin admin] PATH_INFO: `/v3/projects/4b412e8ba93f4308a4643e7d159e1e01/users/ec596cb798ab4dd49f3b912439e5d53e/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:34.707274 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af726e11-d6f8-4f07-ac13-8c96932929b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b412e8ba93f4308a4643e7d159e1e01, user_id=ec596cb798ab4dd49f3b912439e5d53e, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:34.710200 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af726e11-d6f8-4f07-ac13-8c96932929b5 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:34.710644 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5098cb3f-aeb4-41b5-a0f0-ccdc3f9e9f09 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:34.724595 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9612d57-54a8-4b9b-af69-79dff0b92f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:34.725348 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9612d57-54a8-4b9b-af69-79dff0b92f1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:34.725667 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9612d57-54a8-4b9b-af69-79dff0b92f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:34.729472 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9612d57-54a8-4b9b-af69-79dff0b92f1c admin admin] PATH_INFO: `/v3/projects/3ddeeff8356d4ab08a6260ca7750343c/users/4246de3e7143447d934fb7dde53b3dcc/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:34.736198 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9612d57-54a8-4b9b-af69-79dff0b92f1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ddeeff8356d4ab08a6260ca7750343c, user_id=4246de3e7143447d934fb7dde53b3dcc, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:34.742187 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9612d57-54a8-4b9b-af69-79dff0b92f1c admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:34.748364 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:34.781994 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 878/1765] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:42:34 2022] PUT /identity/v3/projects/4b412e8ba93f4308a4643e7d159e1e01/users/ec596cb798ab4dd49f3b912439e5d53e/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:34.781994 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3826fb94-aca2-43b1-a9bc-6e15217701a6 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:34.781994 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3826fb94-aca2-43b1-a9bc-6e15217701a6 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:34.781994 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3826fb94-aca2-43b1-a9bc-6e15217701a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:34.782472 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 888/1766] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:42:34 2022] PUT /identity/v3/projects/3ddeeff8356d4ab08a6260ca7750343c/users/4246de3e7143447d934fb7dde53b3dcc/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:34.782931 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9612d57-54a8-4b9b-af69-79dff0b92f1c admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:34.865970 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-3826fb94-aca2-43b1-a9bc-6e15217701a6 None None] MFA Rules not processed for user `6270e19d6245453ab79555a79532faba`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:34.868676 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-3826fb94-aca2-43b1-a9bc-6e15217701a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:34.921788 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:34.940846 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9612d57-54a8-4b9b-af69-79dff0b92f1c admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:34.954620 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616bd599-607b-4622-9ff8-45ab05d1a714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:34.957080 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616bd599-607b-4622-9ff8-45ab05d1a714 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:34.957655 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616bd599-607b-4622-9ff8-45ab05d1a714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:34.957655 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616bd599-607b-4622-9ff8-45ab05d1a714 admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:34.958096 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616bd599-607b-4622-9ff8-45ab05d1a714 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:34.962394 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616bd599-607b-4622-9ff8-45ab05d1a714 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:35.049235 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 889/1767] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:42:34 2022] POST /identity/v3/users => generated 301 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:35.056807 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84dfb9c-9f05-45cb-9b0e-1d1dd74bdbde None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:35.057096 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84dfb9c-9f05-45cb-9b0e-1d1dd74bdbde None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:35.057392 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84dfb9c-9f05-45cb-9b0e-1d1dd74bdbde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:35.150224 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-f84dfb9c-9f05-45cb-9b0e-1d1dd74bdbde None None] MFA Rules not processed for user `ec596cb798ab4dd49f3b912439e5d53e`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:35.151895 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-f84dfb9c-9f05-45cb-9b0e-1d1dd74bdbde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:35.161676 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 879/1768] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:34 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 388 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:35.162872 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68628afc-df22-4fd3-add4-170666a59459 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:35.165317 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68628afc-df22-4fd3-add4-170666a59459 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:35.165627 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68628afc-df22-4fd3-add4-170666a59459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:35.247945 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-68628afc-df22-4fd3-add4-170666a59459 None None] MFA Rules not processed for user `4246de3e7143447d934fb7dde53b3dcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:35.251185 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-68628afc-df22-4fd3-add4-170666a59459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:35.303102 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 890/1769] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:35.304089 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84dfb9c-9f05-45cb-9b0e-1d1dd74bdbde None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:35.319869 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84dfb9c-9f05-45cb-9b0e-1d1dd74bdbde None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:35.327605 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb48de0-8ed3-4c69-a1c8-fccdf517854a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:35.328437 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb48de0-8ed3-4c69-a1c8-fccdf517854a admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:35.328872 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb48de0-8ed3-4c69-a1c8-fccdf517854a admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:35.329413 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb48de0-8ed3-4c69-a1c8-fccdf517854a admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:35.330279 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb48de0-8ed3-4c69-a1c8-fccdf517854a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:35.332216 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb48de0-8ed3-4c69-a1c8-fccdf517854a admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:35.353470 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 891/1770] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:35 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:35.354467 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb48de0-8ed3-4c69-a1c8-fccdf517854a admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:35.357351 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-9eb48de0-8ed3-4c69-a1c8-fccdf517854a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:35.403831 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 880/1771] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:35 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:35.419048 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68628afc-df22-4fd3-add4-170666a59459 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:35.421818 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-68628afc-df22-4fd3-add4-170666a59459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:35.439759 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68628afc-df22-4fd3-add4-170666a59459 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:35.448225 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef0b8f1-54a6-41c7-9200-3efae7b79b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:35.449039 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0b8f1-54a6-41c7-9200-3efae7b79b4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:35.449565 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0b8f1-54a6-41c7-9200-3efae7b79b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:35.450291 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0b8f1-54a6-41c7-9200-3efae7b79b4b admin admin] PATH_INFO: `/v3/users/aafc618b52ae496299fdf53d2cac771a` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:35.452690 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef0b8f1-54a6-41c7-9200-3efae7b79b4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=aafc618b52ae496299fdf53d2cac771a)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:35.455678 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef0b8f1-54a6-41c7-9200-3efae7b79b4b admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:35.545829 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb48de0-8ed3-4c69-a1c8-fccdf517854a admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:35.581715 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757b281-6a19-441a-aebf-a19b7774ff6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:35.582530 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757b281-6a19-441a-aebf-a19b7774ff6b service nova] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:35.583792 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757b281-6a19-441a-aebf-a19b7774ff6b service nova] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:35.584201 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757b281-6a19-441a-aebf-a19b7774ff6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:35.605562 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5757b281-6a19-441a-aebf-a19b7774ff6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:35.609883 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5757b281-6a19-441a-aebf-a19b7774ff6b service nova] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:35.657003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-5ef0b8f1-54a6-41c7-9200-3efae7b79b4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aafc618b52ae496299fdf53d2cac771a'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:35.659886 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 892/1772] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:35 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:35.660858 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757b281-6a19-441a-aebf-a19b7774ff6b service nova] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:35.664607 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-5ef0b8f1-54a6-41c7-9200-3efae7b79b4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aafc618b52ae496299fdf53d2cac771a'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:35.688719 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757b281-6a19-441a-aebf-a19b7774ff6b service nova] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:35.701301 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-5ef0b8f1-54a6-41c7-9200-3efae7b79b4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aafc618b52ae496299fdf53d2cac771a'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:35.702432 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d03111-322e-4cae-a83a-7e557cfdb3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:35.703167 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d03111-322e-4cae-a83a-7e557cfdb3aa admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:35.704131 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d03111-322e-4cae-a83a-7e557cfdb3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:35.704405 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d03111-322e-4cae-a83a-7e557cfdb3aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:35.705157 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d03111-322e-4cae-a83a-7e557cfdb3aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:35.711996 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d03111-322e-4cae-a83a-7e557cfdb3aa admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:35.721910 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:35.727556 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-5ef0b8f1-54a6-41c7-9200-3efae7b79b4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aafc618b52ae496299fdf53d2cac771a'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:35.758803 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-5ef0b8f1-54a6-41c7-9200-3efae7b79b4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aafc618b52ae496299fdf53d2cac771a'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:35.773017 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 881/1773] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:42:35 2022] DELETE /identity/v3/users/aafc618b52ae496299fdf53d2cac771a => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:35.776223 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef0b8f1-54a6-41c7-9200-3efae7b79b4b admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:35.780149 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-5ef0b8f1-54a6-41c7-9200-3efae7b79b4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:35.841969 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 893/1774] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:42:35 2022] POST /identity/v3/projects => generated 375 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:35.850186 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d03111-322e-4cae-a83a-7e557cfdb3aa admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:35.871715 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-d1d03111-322e-4cae-a83a-7e557cfdb3aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:35.955084 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:36.037941 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef0b8f1-54a6-41c7-9200-3efae7b79b4b admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:36.068667 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d03111-322e-4cae-a83a-7e557cfdb3aa admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:36.074981 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bc9571-e30a-470c-b6e8-8694bd868fe8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:36.079337 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc9571-e30a-470c-b6e8-8694bd868fe8 service cinder] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:36.079734 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc9571-e30a-470c-b6e8-8694bd868fe8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:36.080131 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc9571-e30a-470c-b6e8-8694bd868fe8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:36.087636 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-05bc9571-e30a-470c-b6e8-8694bd868fe8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:36.174839 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bc9571-e30a-470c-b6e8-8694bd868fe8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:36.177855 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bc9571-e30a-470c-b6e8-8694bd868fe8 service cinder] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:36.191986 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 894/1775] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:35 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:36.192967 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bc9571-e30a-470c-b6e8-8694bd868fe8 service cinder] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:36.194517 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-05bc9571-e30a-470c-b6e8-8694bd868fe8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:36.208790 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b6c7e-16b5-43aa-ad6e-3e9c61c48f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:36.209420 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b6c7e-16b5-43aa-ad6e-3e9c61c48f25 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:36.210534 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b6c7e-16b5-43aa-ad6e-3e9c61c48f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:36.210534 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b6c7e-16b5-43aa-ad6e-3e9c61c48f25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:36.210534 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43b6c7e-16b5-43aa-ad6e-3e9c61c48f25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:36.213757 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43b6c7e-16b5-43aa-ad6e-3e9c61c48f25 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:36.235531 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 882/1776] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:35 2022] GET /identity/v3/roles => generated 1420 bytes in 460 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:36.236949 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4315dd0-4919-4a15-8d8a-e00c25f1c748 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:36.237239 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4315dd0-4919-4a15-8d8a-e00c25f1c748 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:36.237635 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4315dd0-4919-4a15-8d8a-e00c25f1c748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:36.284880 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-c4315dd0-4919-4a15-8d8a-e00c25f1c748 None None] MFA Rules not processed for user `35f28006916040c3b988de6c15093166`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:36.286629 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-c4315dd0-4919-4a15-8d8a-e00c25f1c748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:36.394461 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bc9571-e30a-470c-b6e8-8694bd868fe8 service cinder] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:36.420568 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51790678-5ad2-4853-951a-5ec36464ad15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:36.423501 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51790678-5ad2-4853-951a-5ec36464ad15 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:36.426453 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51790678-5ad2-4853-951a-5ec36464ad15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:36.426453 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51790678-5ad2-4853-951a-5ec36464ad15 admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:36.426850 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51790678-5ad2-4853-951a-5ec36464ad15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:36.432425 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51790678-5ad2-4853-951a-5ec36464ad15 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:36.438831 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 883/1777] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:36 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:36.439652 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4315dd0-4919-4a15-8d8a-e00c25f1c748 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:36.479044 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4315dd0-4919-4a15-8d8a-e00c25f1c748 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:36.515442 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20daac95-7030-467f-9842-4238471e50ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:36.515971 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20daac95-7030-467f-9842-4238471e50ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:36.516650 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20daac95-7030-467f-9842-4238471e50ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:36.516794 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 895/1778] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:42:36 2022] POST /identity/v3/users => generated 294 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:36.516883 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20daac95-7030-467f-9842-4238471e50ad admin admin] PATH_INFO: `/v3/projects/82470c9b7483440f9b87d7c034fccfba/users/27348fdcc1834d99969b5fedc54a9fa9/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:36.523452 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51790678-5ad2-4853-951a-5ec36464ad15 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:36.550213 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51790678-5ad2-4853-951a-5ec36464ad15 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:36.561239 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c5f3ef-74d1-4f4d-919a-1829f21df55f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:36.562172 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c5f3ef-74d1-4f4d-919a-1829f21df55f service cinder] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:36.563868 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c5f3ef-74d1-4f4d-919a-1829f21df55f service cinder] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:36.564371 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c5f3ef-74d1-4f4d-919a-1829f21df55f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:36.571036 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-a8c5f3ef-74d1-4f4d-919a-1829f21df55f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:36.582255 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20daac95-7030-467f-9842-4238471e50ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=82470c9b7483440f9b87d7c034fccfba, user_id=27348fdcc1834d99969b5fedc54a9fa9, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:36.609942 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20daac95-7030-467f-9842-4238471e50ad admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:36.640292 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 884/1779] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:42:36 2022] PUT /identity/v3/projects/82470c9b7483440f9b87d7c034fccfba/users/27348fdcc1834d99969b5fedc54a9fa9/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:36.641229 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20daac95-7030-467f-9842-4238471e50ad admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:36.650273 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-20daac95-7030-467f-9842-4238471e50ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:36.676382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:36.679652 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c5f3ef-74d1-4f4d-919a-1829f21df55f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:36.681441 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c5f3ef-74d1-4f4d-919a-1829f21df55f service cinder] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:36.698117 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 896/1780] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:36 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:36.700959 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c5f3ef-74d1-4f4d-919a-1829f21df55f service cinder] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:36.715192 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:36.731039 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c5f3ef-74d1-4f4d-919a-1829f21df55f service cinder] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:36.849810 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20daac95-7030-467f-9842-4238471e50ad admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:36.866204 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d4844-94c4-4606-aecd-3d339956a500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:36.869819 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d4844-94c4-4606-aecd-3d339956a500 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:36.870381 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d4844-94c4-4606-aecd-3d339956a500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:36.870771 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d4844-94c4-4606-aecd-3d339956a500 admin admin] PATH_INFO: `/v3/projects/96d3213bbb054cdf82301bd361be3c36` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:36.873659 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5d4844-94c4-4606-aecd-3d339956a500 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96d3213bbb054cdf82301bd361be3c36)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:36.880577 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5d4844-94c4-4606-aecd-3d339956a500 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:36.933812 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5451b8e9-5d32-4c55-88d8-d19d694de76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:36.940509 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5451b8e9-5d32-4c55-88d8-d19d694de76e admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:36.940797 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5451b8e9-5d32-4c55-88d8-d19d694de76e admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:36.941078 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5451b8e9-5d32-4c55-88d8-d19d694de76e admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:36.944899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5451b8e9-5d32-4c55-88d8-d19d694de76e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:36.946554 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5451b8e9-5d32-4c55-88d8-d19d694de76e admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:36.966278 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 897/1781] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:36 2022] GET /identity/v3/roles => generated 1420 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:36.974421 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a58e9-7b28-4130-9a5a-5a923b1126d3 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:36.979358 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a58e9-7b28-4130-9a5a-5a923b1126d3 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:36.979845 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a58e9-7b28-4130-9a5a-5a923b1126d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:37.037615 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-2f5d4844-94c4-4606-aecd-3d339956a500 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96d3213bbb054cdf82301bd361be3c36', 'request_id': 'req-2f5d4844-94c4-4606-aecd-3d339956a500'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:37.055521 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-2f5d4844-94c4-4606-aecd-3d339956a500 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96d3213bbb054cdf82301bd361be3c36', 'request_id': 'req-2f5d4844-94c4-4606-aecd-3d339956a500'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:37.068095 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-2f5d4844-94c4-4606-aecd-3d339956a500 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96d3213bbb054cdf82301bd361be3c36', 'request_id': 'req-2f5d4844-94c4-4606-aecd-3d339956a500'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:37.088816 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-2f5d4844-94c4-4606-aecd-3d339956a500 admin admin] The token cache is being invalidate because project 96d3213bbb054cdf82301bd361be3c36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:42:37.089197 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-2f5d4844-94c4-4606-aecd-3d339956a500 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:37.098430 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-2f5d4844-94c4-4606-aecd-3d339956a500 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:37.104635 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 885/1782] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:42:36 2022] DELETE /identity/v3/projects/96d3213bbb054cdf82301bd361be3c36 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:37.106809 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d4857-e86b-429a-b676-67a098415b98 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:37.109552 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-614a58e9-7b28-4130-9a5a-5a923b1126d3 None None] MFA Rules not processed for user `f60bee471e954932a928ab2cf7d66c82`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:37.110797 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-614a58e9-7b28-4130-9a5a-5a923b1126d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:37.110917 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d4857-e86b-429a-b676-67a098415b98 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:37.111168 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d4857-e86b-429a-b676-67a098415b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:37.176795 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-bf4d4857-e86b-429a-b676-67a098415b98 None None] MFA Rules not processed for user `27348fdcc1834d99969b5fedc54a9fa9`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:37.185740 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-bf4d4857-e86b-429a-b676-67a098415b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:37.318732 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 898/1783] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Aug 30 14:42:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:37.319807 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614a58e9-7b28-4130-9a5a-5a923b1126d3 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:37.325023 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-614a58e9-7b28-4130-9a5a-5a923b1126d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:37.416554 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 886/1784] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:37 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:37.417358 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d4857-e86b-429a-b676-67a098415b98 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:37.422513 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-bf4d4857-e86b-429a-b676-67a098415b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:37.505634 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:37.521442 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614a58e9-7b28-4130-9a5a-5a923b1126d3 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:37.528551 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c9fb02-0977-446e-af80-c87ed00047af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:37.537598 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c9fb02-0977-446e-af80-c87ed00047af admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:37.538162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c9fb02-0977-446e-af80-c87ed00047af admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:37.538584 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c9fb02-0977-446e-af80-c87ed00047af admin admin] PATH_INFO: `/v3/projects/db900377140041a2a07240729b467475/users/54743e5571fe487c8f815461cb699c06/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:37.563642 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c9fb02-0977-446e-af80-c87ed00047af admin admin] RBAC: Authorizing `identity:create_grant(project_id=db900377140041a2a07240729b467475, user_id=54743e5571fe487c8f815461cb699c06, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:37.575378 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c9fb02-0977-446e-af80-c87ed00047af admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:37.580034 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:37.611950 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 899/1785] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:42:37 2022] PUT /identity/v3/projects/db900377140041a2a07240729b467475/users/54743e5571fe487c8f815461cb699c06/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:37.615751 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c9fb02-0977-446e-af80-c87ed00047af admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:37.621508 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-f7c9fb02-0977-446e-af80-c87ed00047af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:37.632634 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d4857-e86b-429a-b676-67a098415b98 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:37.817148 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c9fb02-0977-446e-af80-c87ed00047af admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:37.842017 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48531486-3859-4270-83c8-a9a731b76ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:37.850847 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48531486-3859-4270-83c8-a9a731b76ea7 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:37.851748 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48531486-3859-4270-83c8-a9a731b76ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:37.852861 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48531486-3859-4270-83c8-a9a731b76ea7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:37.854483 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48531486-3859-4270-83c8-a9a731b76ea7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:37.867376 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48531486-3859-4270-83c8-a9a731b76ea7 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:37.880068 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475e2f4d-7b0f-47d0-a5f5-56b2f1d811d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:37.880730 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e2f4d-7b0f-47d0-a5f5-56b2f1d811d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:37.881015 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e2f4d-7b0f-47d0-a5f5-56b2f1d811d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:37.889389 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e2f4d-7b0f-47d0-a5f5-56b2f1d811d3 admin admin] PATH_INFO: `/v3/users/c06c0ad183804721a047e34e37a115de` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:37.894109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475e2f4d-7b0f-47d0-a5f5-56b2f1d811d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c06c0ad183804721a047e34e37a115de)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:37.896156 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475e2f4d-7b0f-47d0-a5f5-56b2f1d811d3 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:37.950809 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 900/1786] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:42:37 2022] POST /identity/v3/projects => generated 401 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:37.953127 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48531486-3859-4270-83c8-a9a731b76ea7 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:38.035442 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-475e2f4d-7b0f-47d0-a5f5-56b2f1d811d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c06c0ad183804721a047e34e37a115de'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:38.036490 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-475e2f4d-7b0f-47d0-a5f5-56b2f1d811d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c06c0ad183804721a047e34e37a115de'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:38.059158 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-475e2f4d-7b0f-47d0-a5f5-56b2f1d811d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c06c0ad183804721a047e34e37a115de'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:38.098590 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-475e2f4d-7b0f-47d0-a5f5-56b2f1d811d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c06c0ad183804721a047e34e37a115de'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:38.129647 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-475e2f4d-7b0f-47d0-a5f5-56b2f1d811d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c06c0ad183804721a047e34e37a115de'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:38.142763 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48531486-3859-4270-83c8-a9a731b76ea7 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:38.146986 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 887/1787] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:42:37 2022] DELETE /identity/v3/users/c06c0ad183804721a047e34e37a115de => generated 0 bytes in 730 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:38.148062 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475e2f4d-7b0f-47d0-a5f5-56b2f1d811d3 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:38.150292 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-475e2f4d-7b0f-47d0-a5f5-56b2f1d811d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:38.363059 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3130040-6661-46bc-996d-3bdd9c8f22f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:38.365295 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3130040-6661-46bc-996d-3bdd9c8f22f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:38.367583 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3130040-6661-46bc-996d-3bdd9c8f22f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:38.369460 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3130040-6661-46bc-996d-3bdd9c8f22f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:38.372522 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3130040-6661-46bc-996d-3bdd9c8f22f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:38.379197 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3130040-6661-46bc-996d-3bdd9c8f22f8 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:38.404307 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:38.429105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 901/1788] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:37 2022] GET /identity/v3/roles => generated 1420 bytes in 476 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:38.430227 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3130040-6661-46bc-996d-3bdd9c8f22f8 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:38.435022 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-a3130040-6661-46bc-996d-3bdd9c8f22f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:38.464212 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3130040-6661-46bc-996d-3bdd9c8f22f8 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:38.483071 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475e2f4d-7b0f-47d0-a5f5-56b2f1d811d3 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:38.488254 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ed1315-e0e5-4577-9cd7-35cc8238f6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:38.488930 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed1315-e0e5-4577-9cd7-35cc8238f6e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:38.491046 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed1315-e0e5-4577-9cd7-35cc8238f6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:38.491407 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed1315-e0e5-4577-9cd7-35cc8238f6e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:38.492077 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ed1315-e0e5-4577-9cd7-35cc8238f6e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:38.499768 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ed1315-e0e5-4577-9cd7-35cc8238f6e8 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:38.507782 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6fd156-50b6-4180-80ed-a55229e95f1e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a18b13e3708144e1b5e8df80ec3a4e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:38.508636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6fd156-50b6-4180-80ed-a55229e95f1e service swift] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:38.509349 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6fd156-50b6-4180-80ed-a55229e95f1e service swift] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:38.509956 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6fd156-50b6-4180-80ed-a55229e95f1e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:38.518710 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-9c6fd156-50b6-4180-80ed-a55229e95f1e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:38.578477 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 902/1789] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:42:38 2022] POST /identity/v3/users => generated 307 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:38.581452 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1defb0c-5ab1-4bac-a7d5-123ab9610b9e None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:38.582375 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1defb0c-5ab1-4bac-a7d5-123ab9610b9e None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:38.583413 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1defb0c-5ab1-4bac-a7d5-123ab9610b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:38.618228 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:38.662477 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-b1defb0c-5ab1-4bac-a7d5-123ab9610b9e None None] MFA Rules not processed for user `ca9bafc5ff4e43719f37e62cad946cf7`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:38.673555 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-b1defb0c-5ab1-4bac-a7d5-123ab9610b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:38.759901 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6fd156-50b6-4180-80ed-a55229e95f1e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:38.761544 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6fd156-50b6-4180-80ed-a55229e95f1e service swift] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:38.809827 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 903/1790] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:38.815407 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1defb0c-5ab1-4bac-a7d5-123ab9610b9e None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:38.815887 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-b1defb0c-5ab1-4bac-a7d5-123ab9610b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:38.833407 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 888/1791] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Aug 30 14:42:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 685 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 14:42:38.836391 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6fd156-50b6-4180-80ed-a55229e95f1e service swift] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:38.874303 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6fd156-50b6-4180-80ed-a55229e95f1e service swift] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:38.883749 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1defb0c-5ab1-4bac-a7d5-123ab9610b9e None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:38.899410 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7203c64d-3749-44df-a518-db5105fcc428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:38.900082 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203c64d-3749-44df-a518-db5105fcc428 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:38.904176 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203c64d-3749-44df-a518-db5105fcc428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:38.904507 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203c64d-3749-44df-a518-db5105fcc428 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:38.907538 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff8d234-155e-46df-9e85-d8c4194b9221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:38.909651 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff8d234-155e-46df-9e85-d8c4194b9221 admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:38.911128 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff8d234-155e-46df-9e85-d8c4194b9221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:38.911648 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff8d234-155e-46df-9e85-d8c4194b9221 admin admin] PATH_INFO: `/v3/projects/db900377140041a2a07240729b467475/users/54743e5571fe487c8f815461cb699c06/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:38.918081 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff8d234-155e-46df-9e85-d8c4194b9221 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db900377140041a2a07240729b467475, user_id=54743e5571fe487c8f815461cb699c06, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:38.920413 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7203c64d-3749-44df-a518-db5105fcc428 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:38.923522 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7203c64d-3749-44df-a518-db5105fcc428 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:38.959426 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff8d234-155e-46df-9e85-d8c4194b9221 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:39.009635 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 889/1792] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:38 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:39.016567 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7203c64d-3749-44df-a518-db5105fcc428 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:39.037813 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 904/1793] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:42:38 2022] PUT /identity/v3/projects/db900377140041a2a07240729b467475/users/54743e5571fe487c8f815461cb699c06/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:39.042513 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec7911-e8e6-4893-95bc-9e45a1d48dfb None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:39.043016 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec7911-e8e6-4893-95bc-9e45a1d48dfb None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:39.043533 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec7911-e8e6-4893-95bc-9e45a1d48dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:39.103150 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7203c64d-3749-44df-a518-db5105fcc428 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:39.163407 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-79ec7911-e8e6-4893-95bc-9e45a1d48dfb None None] MFA Rules not processed for user `54743e5571fe487c8f815461cb699c06`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:39.164697 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-79ec7911-e8e6-4893-95bc-9e45a1d48dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:39.377955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb97ead-407b-435c-8182-5506be51834c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:39.378987 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb97ead-407b-435c-8182-5506be51834c admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:39.379490 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb97ead-407b-435c-8182-5506be51834c admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:39.379957 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb97ead-407b-435c-8182-5506be51834c admin admin] PATH_INFO: `/v3/projects/768a2489be74473b9cf346bdf346948e/users/f4ab48e03d4640e586b51c126e1f9b9a/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:39.403407 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb97ead-407b-435c-8182-5506be51834c admin admin] RBAC: Authorizing `identity:create_grant(project_id=768a2489be74473b9cf346bdf346948e, user_id=f4ab48e03d4640e586b51c126e1f9b9a, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:39.408585 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb97ead-407b-435c-8182-5506be51834c admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:39.459095 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 890/1794] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:42:39 2022] PUT /identity/v3/projects/768a2489be74473b9cf346bdf346948e/users/f4ab48e03d4640e586b51c126e1f9b9a/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:39.463869 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb97ead-407b-435c-8182-5506be51834c admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:39.558376 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 905/1795] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:39 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 520 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:39.572254 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ec7911-e8e6-4893-95bc-9e45a1d48dfb None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:39.574766 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-79ec7911-e8e6-4893-95bc-9e45a1d48dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:39.709967 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ec7911-e8e6-4893-95bc-9e45a1d48dfb None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:39.729884 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:39.783974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb97ead-407b-435c-8182-5506be51834c admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:39.800725 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0964e095-2177-49f8-8ce5-9b2c2e946793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:39.801965 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964e095-2177-49f8-8ce5-9b2c2e946793 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:39.805459 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964e095-2177-49f8-8ce5-9b2c2e946793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:39.805459 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964e095-2177-49f8-8ce5-9b2c2e946793 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:39.805720 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0964e095-2177-49f8-8ce5-9b2c2e946793 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:39.809346 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0964e095-2177-49f8-8ce5-9b2c2e946793 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:39.825499 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:39.869453 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 891/1796] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:39 2022] GET /identity/v3/roles => generated 1420 bytes in 407 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:39.875486 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0964e095-2177-49f8-8ce5-9b2c2e946793 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:39.883466 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3798d6-7e7a-4180-b318-7ce0d6b58cc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:39.884214 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3798d6-7e7a-4180-b318-7ce0d6b58cc8 service cinder] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:39.884471 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3798d6-7e7a-4180-b318-7ce0d6b58cc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:39.884757 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3798d6-7e7a-4180-b318-7ce0d6b58cc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:39.899286 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-3a3798d6-7e7a-4180-b318-7ce0d6b58cc8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:39.915382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0964e095-2177-49f8-8ce5-9b2c2e946793 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:39.930287 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0138fc23-faf0-49d7-97c0-5134ca97377d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:39.934541 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0138fc23-faf0-49d7-97c0-5134ca97377d admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:39.934848 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0138fc23-faf0-49d7-97c0-5134ca97377d admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:39.937350 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0138fc23-faf0-49d7-97c0-5134ca97377d admin admin] PATH_INFO: `/v3/projects/768a2489be74473b9cf346bdf346948e/users/f4ab48e03d4640e586b51c126e1f9b9a/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:39.944389 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0138fc23-faf0-49d7-97c0-5134ca97377d admin admin] RBAC: Authorizing `identity:create_grant(project_id=768a2489be74473b9cf346bdf346948e, user_id=f4ab48e03d4640e586b51c126e1f9b9a, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:39.950871 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0138fc23-faf0-49d7-97c0-5134ca97377d admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:40.001437 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 892/1797] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:42:39 2022] PUT /identity/v3/projects/768a2489be74473b9cf346bdf346948e/users/f4ab48e03d4640e586b51c126e1f9b9a/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:40.003626 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec25953-5eee-47d8-8503-d313dcf215c5 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:40.003911 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec25953-5eee-47d8-8503-d313dcf215c5 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:40.004182 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec25953-5eee-47d8-8503-d313dcf215c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:40.078451 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3798d6-7e7a-4180-b318-7ce0d6b58cc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:40.080819 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3798d6-7e7a-4180-b318-7ce0d6b58cc8 service cinder] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:40.099573 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-7ec25953-5eee-47d8-8503-d313dcf215c5 None None] MFA Rules not processed for user `f4ab48e03d4640e586b51c126e1f9b9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:40.109335 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 906/1798] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:40.113210 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-7ec25953-5eee-47d8-8503-d313dcf215c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:40.450074 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 893/1799] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:40 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 444 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:40.459879 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3798d6-7e7a-4180-b318-7ce0d6b58cc8 service cinder] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:40.462021 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-3a3798d6-7e7a-4180-b318-7ce0d6b58cc8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:40.475141 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3798d6-7e7a-4180-b318-7ce0d6b58cc8 service cinder] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:40.614407 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2af81b8-e9e5-45b2-87a3-f71372d5d262 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:40.615342 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af81b8-e9e5-45b2-87a3-f71372d5d262 service cinder] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:40.615784 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af81b8-e9e5-45b2-87a3-f71372d5d262 service cinder] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:40.616217 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af81b8-e9e5-45b2-87a3-f71372d5d262 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:40.645707 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2af81b8-e9e5-45b2-87a3-f71372d5d262 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:40.647181 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2af81b8-e9e5-45b2-87a3-f71372d5d262 service cinder] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:40.681541 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 907/1800] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:40 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:40.866404 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec25953-5eee-47d8-8503-d313dcf215c5 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:40.868275 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-7ec25953-5eee-47d8-8503-d313dcf215c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:41.045747 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2af81b8-e9e5-45b2-87a3-f71372d5d262 service cinder] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:41.055601 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2af81b8-e9e5-45b2-87a3-f71372d5d262 service cinder] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:41.087784 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba1a942-6f86-4dd8-896d-f6f4c2117b81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:41.088471 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba1a942-6f86-4dd8-896d-f6f4c2117b81 service cinder] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:41.088681 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba1a942-6f86-4dd8-896d-f6f4c2117b81 service cinder] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:41.088964 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba1a942-6f86-4dd8-896d-f6f4c2117b81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:41.110686 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-cba1a942-6f86-4dd8-896d-f6f4c2117b81 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:41.299535 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec25953-5eee-47d8-8503-d313dcf215c5 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:41.324852 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412aab9e-b939-4200-ae41-808cbc70f713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:41.325553 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412aab9e-b939-4200-ae41-808cbc70f713 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:41.325838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412aab9e-b939-4200-ae41-808cbc70f713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:41.326112 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412aab9e-b939-4200-ae41-808cbc70f713 admin admin] PATH_INFO: `/v3/projects/81bd4c6161a841efa7813d962322ff5d` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:41.337335 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412aab9e-b939-4200-ae41-808cbc70f713 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81bd4c6161a841efa7813d962322ff5d)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:41.339379 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412aab9e-b939-4200-ae41-808cbc70f713 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:41.481137 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba1a942-6f86-4dd8-896d-f6f4c2117b81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:41.482606 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba1a942-6f86-4dd8-896d-f6f4c2117b81 service cinder] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:41.528043 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 908/1801] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:41 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:42:41.612054 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-412aab9e-b939-4200-ae41-808cbc70f713 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81bd4c6161a841efa7813d962322ff5d', 'request_id': 'req-412aab9e-b939-4200-ae41-808cbc70f713'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:41.638084 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-412aab9e-b939-4200-ae41-808cbc70f713 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81bd4c6161a841efa7813d962322ff5d', 'request_id': 'req-412aab9e-b939-4200-ae41-808cbc70f713'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:41.654179 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:41.666111 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-412aab9e-b939-4200-ae41-808cbc70f713 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81bd4c6161a841efa7813d962322ff5d', 'request_id': 'req-412aab9e-b939-4200-ae41-808cbc70f713'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:41.715812 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-412aab9e-b939-4200-ae41-808cbc70f713 admin admin] The token cache is being invalidate because project 81bd4c6161a841efa7813d962322ff5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:42:41.719620 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-412aab9e-b939-4200-ae41-808cbc70f713 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:41.721551 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-412aab9e-b939-4200-ae41-808cbc70f713 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:41.728771 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 894/1802] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:42:40 2022] DELETE /identity/v3/projects/81bd4c6161a841efa7813d962322ff5d => generated 0 bytes in 863 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:41.735066 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1afb6-7f30-4272-ab1b-f38649a34137 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:41.735380 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1afb6-7f30-4272-ab1b-f38649a34137 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:41.735656 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1afb6-7f30-4272-ab1b-f38649a34137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:41.813448 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-8cb1afb6-7f30-4272-ab1b-f38649a34137 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:41.814760 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-8cb1afb6-7f30-4272-ab1b-f38649a34137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:41.965406 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:42.109260 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 909/1803] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:42.117923 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0ca5b-c869-49e6-bc48-d6c4bbfdb74f None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:42.118260 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0ca5b-c869-49e6-bc48-d6c4bbfdb74f None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:42.118488 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0ca5b-c869-49e6-bc48-d6c4bbfdb74f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:42.174632 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-74c0ca5b-c869-49e6-bc48-d6c4bbfdb74f None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:42.176848 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-74c0ca5b-c869-49e6-bc48-d6c4bbfdb74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:42.192550 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1afb6-7f30-4272-ab1b-f38649a34137 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:42.216514 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 895/1804] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:42.218745 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1afb6-7f30-4272-ab1b-f38649a34137 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:42.225589 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d3f514-0cb5-487f-b84d-ae55411c44e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:42.226415 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d3f514-0cb5-487f-b84d-ae55411c44e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:42.227004 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d3f514-0cb5-487f-b84d-ae55411c44e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:42.227477 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d3f514-0cb5-487f-b84d-ae55411c44e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:42.228415 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d3f514-0cb5-487f-b84d-ae55411c44e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:42:42.228933 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d3f514-0cb5-487f-b84d-ae55411c44e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:42.230722 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d3f514-0cb5-487f-b84d-ae55411c44e5 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:42.254016 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0ca5b-c869-49e6-bc48-d6c4bbfdb74f None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:42.259652 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 910/1805] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 14:42:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:42:42.279093 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0ca5b-c869-49e6-bc48-d6c4bbfdb74f None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:42.286305 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d203c3f6-a0b8-44e2-a972-7bdacd100259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:42.288006 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d203c3f6-a0b8-44e2-a972-7bdacd100259 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:42.288494 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d203c3f6-a0b8-44e2-a972-7bdacd100259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:42.289150 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d203c3f6-a0b8-44e2-a972-7bdacd100259 admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:42.290860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d203c3f6-a0b8-44e2-a972-7bdacd100259 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:42.293782 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d203c3f6-a0b8-44e2-a972-7bdacd100259 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:42.388870 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d3f514-0cb5-487f-b84d-ae55411c44e5 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:42.399462 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 896/1806] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:42:42 2022] POST /identity/v3/projects => generated 357 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:42.420134 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d3f514-0cb5-487f-b84d-ae55411c44e5 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:42.630932 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a7000f-29da-42e2-807a-7bc55511c9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:42.633587 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a7000f-29da-42e2-807a-7bc55511c9a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:42.634312 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a7000f-29da-42e2-807a-7bc55511c9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:42.635235 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a7000f-29da-42e2-807a-7bc55511c9a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:42.636058 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a7000f-29da-42e2-807a-7bc55511c9a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:42.638560 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a7000f-29da-42e2-807a-7bc55511c9a9 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:42.713769 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 911/1807] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:42:42 2022] POST /identity/v3/users => generated 279 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:42.717956 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d203c3f6-a0b8-44e2-a972-7bdacd100259 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:42.741476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d203c3f6-a0b8-44e2-a972-7bdacd100259 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:42.757552 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd227c2-5de4-42dc-a6d4-1d0d1021fa39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:42.758292 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd227c2-5de4-42dc-a6d4-1d0d1021fa39 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:42.758726 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd227c2-5de4-42dc-a6d4-1d0d1021fa39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:42.759346 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd227c2-5de4-42dc-a6d4-1d0d1021fa39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:42.760032 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd227c2-5de4-42dc-a6d4-1d0d1021fa39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:42.766368 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd227c2-5de4-42dc-a6d4-1d0d1021fa39 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:42.793489 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 897/1808] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:42.798016 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a7000f-29da-42e2-807a-7bc55511c9a9 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:42.830796 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a7000f-29da-42e2-807a-7bc55511c9a9 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:42.861189 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b77073-bcbd-4e40-a672-c42141fd18a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:42.862380 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b77073-bcbd-4e40-a672-c42141fd18a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:42.863394 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b77073-bcbd-4e40-a672-c42141fd18a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:42.863888 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b77073-bcbd-4e40-a672-c42141fd18a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:42.866357 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b77073-bcbd-4e40-a672-c42141fd18a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:42.871681 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b77073-bcbd-4e40-a672-c42141fd18a2 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:42.915058 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 912/1809] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:42.916320 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd227c2-5de4-42dc-a6d4-1d0d1021fa39 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:42.928651 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd227c2-5de4-42dc-a6d4-1d0d1021fa39 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:42.945850 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c090bb4e-a6e8-4f33-a978-8052ba6d416c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:42.946867 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090bb4e-a6e8-4f33-a978-8052ba6d416c admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:42.947355 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090bb4e-a6e8-4f33-a978-8052ba6d416c admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:42.948897 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090bb4e-a6e8-4f33-a978-8052ba6d416c admin admin] PATH_INFO: `/v3/projects/05aae87b4d6d4b30b43b263141a33007/users/2105fe65015f4ae9bbd025d7725b68ea/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:42.970013 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c090bb4e-a6e8-4f33-a978-8052ba6d416c admin admin] RBAC: Authorizing `identity:create_grant(project_id=05aae87b4d6d4b30b43b263141a33007, user_id=2105fe65015f4ae9bbd025d7725b68ea, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:42.973018 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c090bb4e-a6e8-4f33-a978-8052ba6d416c admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:43.001590 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 898/1810] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:42:42 2022] PUT /identity/v3/projects/05aae87b4d6d4b30b43b263141a33007/users/2105fe65015f4ae9bbd025d7725b68ea/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:43.013232 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f43e5a-4641-48ed-b0a6-cc42524ae796 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:43.013513 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f43e5a-4641-48ed-b0a6-cc42524ae796 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:43.013953 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f43e5a-4641-48ed-b0a6-cc42524ae796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:43.056070 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-76f43e5a-4641-48ed-b0a6-cc42524ae796 None None] MFA Rules not processed for user `2105fe65015f4ae9bbd025d7725b68ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:43.057545 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-76f43e5a-4641-48ed-b0a6-cc42524ae796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:43.147252 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 913/1811] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:43 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:43.157279 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c090bb4e-a6e8-4f33-a978-8052ba6d416c admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:43.291154 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c090bb4e-a6e8-4f33-a978-8052ba6d416c admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:43.301563 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ae9072-ffb7-45d2-b8ed-919a8f158faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:43.302798 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae9072-ffb7-45d2-b8ed-919a8f158faa admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:43.303243 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae9072-ffb7-45d2-b8ed-919a8f158faa admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:43.303885 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae9072-ffb7-45d2-b8ed-919a8f158faa admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:43.304964 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ae9072-ffb7-45d2-b8ed-919a8f158faa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:43.307542 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ae9072-ffb7-45d2-b8ed-919a8f158faa admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:43.352259 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 899/1812] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:42:43 2022] POST /identity/v3/projects => generated 357 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:43.356047 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f43e5a-4641-48ed-b0a6-cc42524ae796 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:43.380685 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f43e5a-4641-48ed-b0a6-cc42524ae796 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:43.415531 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:43.457711 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d84966-0773-40c8-ab51-d4c1b50cb441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:43.458468 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d84966-0773-40c8-ab51-d4c1b50cb441 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:43.458835 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d84966-0773-40c8-ab51-d4c1b50cb441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:43.459184 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d84966-0773-40c8-ab51-d4c1b50cb441 admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:43.459945 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d84966-0773-40c8-ab51-d4c1b50cb441 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:43.462183 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d84966-0773-40c8-ab51-d4c1b50cb441 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:43.494065 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 914/1813] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:42:43 2022] POST /identity/v3/users => generated 285 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:43.495789 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ae9072-ffb7-45d2-b8ed-919a8f158faa admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:43.507497 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ae9072-ffb7-45d2-b8ed-919a8f158faa admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:43.512685 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9240b379-b14d-41a4-8d0d-6e4743c639cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:43.513377 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240b379-b14d-41a4-8d0d-6e4743c639cf admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:43.513955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240b379-b14d-41a4-8d0d-6e4743c639cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:43.515059 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240b379-b14d-41a4-8d0d-6e4743c639cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:43.515952 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9240b379-b14d-41a4-8d0d-6e4743c639cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:43.517656 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9240b379-b14d-41a4-8d0d-6e4743c639cf admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:43.531097 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 900/1814] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:43.534998 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d84966-0773-40c8-ab51-d4c1b50cb441 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:43.552535 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d84966-0773-40c8-ab51-d4c1b50cb441 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:43.557668 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab93de37-20eb-4f81-8ee6-6e5d06982895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:43.558455 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93de37-20eb-4f81-8ee6-6e5d06982895 admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:43.558806 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93de37-20eb-4f81-8ee6-6e5d06982895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:43.559147 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93de37-20eb-4f81-8ee6-6e5d06982895 admin admin] PATH_INFO: `/v3/projects/c81a7c0942ef49e8825bdf58bc33a254/users/4338ff86945549d8916747a3605eca5b/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:43.578230 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab93de37-20eb-4f81-8ee6-6e5d06982895 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c81a7c0942ef49e8825bdf58bc33a254, user_id=4338ff86945549d8916747a3605eca5b, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:43.581031 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab93de37-20eb-4f81-8ee6-6e5d06982895 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:43.597098 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 915/1815] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:42:43 2022] PUT /identity/v3/projects/c81a7c0942ef49e8825bdf58bc33a254/users/4338ff86945549d8916747a3605eca5b/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:43.601524 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9240b379-b14d-41a4-8d0d-6e4743c639cf admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:43.612311 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9240b379-b14d-41a4-8d0d-6e4743c639cf admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:43.681714 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4135b08-80a6-4349-bc78-fb690969966b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:43.682390 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4135b08-80a6-4349-bc78-fb690969966b admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:43.682633 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4135b08-80a6-4349-bc78-fb690969966b admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:43.682901 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4135b08-80a6-4349-bc78-fb690969966b admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:43.683590 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4135b08-80a6-4349-bc78-fb690969966b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:43.685190 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4135b08-80a6-4349-bc78-fb690969966b admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:43.686659 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:43.697976 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 901/1816] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:43 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:43.703290 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab93de37-20eb-4f81-8ee6-6e5d06982895 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:43.720771 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab93de37-20eb-4f81-8ee6-6e5d06982895 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:43.727051 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6895275f-ce97-4a57-a7ae-f13e77ddd127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:43.727752 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895275f-ce97-4a57-a7ae-f13e77ddd127 admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:43.728033 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895275f-ce97-4a57-a7ae-f13e77ddd127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:43.728306 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895275f-ce97-4a57-a7ae-f13e77ddd127 admin admin] PATH_INFO: `/v3/projects/c81a7c0942ef49e8825bdf58bc33a254/users/4338ff86945549d8916747a3605eca5b/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:43.730234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6895275f-ce97-4a57-a7ae-f13e77ddd127 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c81a7c0942ef49e8825bdf58bc33a254, user_id=4338ff86945549d8916747a3605eca5b, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:43.732894 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6895275f-ce97-4a57-a7ae-f13e77ddd127 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:43.747273 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 916/1817] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:42:43 2022] PUT /identity/v3/projects/c81a7c0942ef49e8825bdf58bc33a254/users/4338ff86945549d8916747a3605eca5b/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:42:43.752976 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae93bf-265f-45e2-afde-49ed4860ab45 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:43.753277 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae93bf-265f-45e2-afde-49ed4860ab45 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:43.753577 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae93bf-265f-45e2-afde-49ed4860ab45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:43.758763 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0588d7-af19-4353-8e73-5cdd0b2238dd None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:43.759398 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0588d7-af19-4353-8e73-5cdd0b2238dd None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:43.759777 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0588d7-af19-4353-8e73-5cdd0b2238dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:43.781412 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-e4ae93bf-265f-45e2-afde-49ed4860ab45 None None] MFA Rules not processed for user `6270e19d6245453ab79555a79532faba`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:43.783252 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-e4ae93bf-265f-45e2-afde-49ed4860ab45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:43.804181 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-3f0588d7-af19-4353-8e73-5cdd0b2238dd None None] MFA Rules not processed for user `4338ff86945549d8916747a3605eca5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:43.805669 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-3f0588d7-af19-4353-8e73-5cdd0b2238dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:43.861099 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 902/1818] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:43 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:43.933788 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 917/1819] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:43 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:43.945423 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ae93bf-265f-45e2-afde-49ed4860ab45 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:43.946837 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-e4ae93bf-265f-45e2-afde-49ed4860ab45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:44.128873 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ae93bf-265f-45e2-afde-49ed4860ab45 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:44.132860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dac4e7-e08d-40a3-a55e-33212c6f34c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:44.133475 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dac4e7-e08d-40a3-a55e-33212c6f34c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:44.133754 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dac4e7-e08d-40a3-a55e-33212c6f34c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:44.134027 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dac4e7-e08d-40a3-a55e-33212c6f34c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:44.158210 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dac4e7-e08d-40a3-a55e-33212c6f34c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:44.159724 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dac4e7-e08d-40a3-a55e-33212c6f34c0 service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:44.171274 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 903/1820] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:43 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:44.279026 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0588d7-af19-4353-8e73-5cdd0b2238dd None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:44.297918 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0588d7-af19-4353-8e73-5cdd0b2238dd None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:44.302029 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25709c2d-4261-4a71-98e5-7a9950b2def0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:44.302717 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25709c2d-4261-4a71-98e5-7a9950b2def0 service cinder] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:44.303060 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25709c2d-4261-4a71-98e5-7a9950b2def0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:44.303444 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25709c2d-4261-4a71-98e5-7a9950b2def0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:44.393711 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25709c2d-4261-4a71-98e5-7a9950b2def0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:44.395438 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25709c2d-4261-4a71-98e5-7a9950b2def0 service cinder] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:44.408535 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 918/1821] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:44 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:45.941220 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dac4e7-e08d-40a3-a55e-33212c6f34c0 service cinder] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:45.947161 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-00dac4e7-e08d-40a3-a55e-33212c6f34c0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:46.058599 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dac4e7-e08d-40a3-a55e-33212c6f34c0 service cinder] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:46.065560 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adc61a9-d472-4cb6-bf6c-7f2954fb3c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:46.066387 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adc61a9-d472-4cb6-bf6c-7f2954fb3c49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:46.066778 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adc61a9-d472-4cb6-bf6c-7f2954fb3c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:46.067158 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adc61a9-d472-4cb6-bf6c-7f2954fb3c49 admin admin] PATH_INFO: `/v3/users/ec596cb798ab4dd49f3b912439e5d53e` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:46.069018 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adc61a9-d472-4cb6-bf6c-7f2954fb3c49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec596cb798ab4dd49f3b912439e5d53e)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:46.071526 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adc61a9-d472-4cb6-bf6c-7f2954fb3c49 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:46.146769 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:46.153469 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-3adc61a9-d472-4cb6-bf6c-7f2954fb3c49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec596cb798ab4dd49f3b912439e5d53e'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:46.159227 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-3adc61a9-d472-4cb6-bf6c-7f2954fb3c49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec596cb798ab4dd49f3b912439e5d53e'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:46.178607 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-3adc61a9-d472-4cb6-bf6c-7f2954fb3c49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec596cb798ab4dd49f3b912439e5d53e'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:46.194806 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-3adc61a9-d472-4cb6-bf6c-7f2954fb3c49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec596cb798ab4dd49f3b912439e5d53e'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:46.205001 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-3adc61a9-d472-4cb6-bf6c-7f2954fb3c49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec596cb798ab4dd49f3b912439e5d53e'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:46.221412 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 904/1822] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:42:45 2022] DELETE /identity/v3/users/ec596cb798ab4dd49f3b912439e5d53e => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:46.227597 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f339e-bc28-44f4-a1ae-a05ff704192f None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:46.228117 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f339e-bc28-44f4-a1ae-a05ff704192f None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:46.228208 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f339e-bc28-44f4-a1ae-a05ff704192f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:46.242073 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:46.259070 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-1f5f339e-bc28-44f4-a1ae-a05ff704192f None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:46.262222 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-1f5f339e-bc28-44f4-a1ae-a05ff704192f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:46.383852 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 919/1823] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:46.390904 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adc61a9-d472-4cb6-bf6c-7f2954fb3c49 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:46.393191 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-3adc61a9-d472-4cb6-bf6c-7f2954fb3c49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:46.518989 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adc61a9-d472-4cb6-bf6c-7f2954fb3c49 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:46.523068 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b131e3-e22f-4691-8ea0-226b88af2bdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:46.524067 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b131e3-e22f-4691-8ea0-226b88af2bdf service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:46.524347 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b131e3-e22f-4691-8ea0-226b88af2bdf service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:46.524687 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b131e3-e22f-4691-8ea0-226b88af2bdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:46.546813 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b131e3-e22f-4691-8ea0-226b88af2bdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:46.553196 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b131e3-e22f-4691-8ea0-226b88af2bdf service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:46.575716 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 905/1824] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:47.289905 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f339e-bc28-44f4-a1ae-a05ff704192f None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:47.293033 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-1f5f339e-bc28-44f4-a1ae-a05ff704192f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:47.342627 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f339e-bc28-44f4-a1ae-a05ff704192f None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:47.354158 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c2d77a-83cf-45e6-ad02-fec541fe49e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:47.354847 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2d77a-83cf-45e6-ad02-fec541fe49e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:47.355127 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2d77a-83cf-45e6-ad02-fec541fe49e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:47.355444 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2d77a-83cf-45e6-ad02-fec541fe49e5 admin admin] PATH_INFO: `/v3/projects/4b412e8ba93f4308a4643e7d159e1e01` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:47.357011 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c2d77a-83cf-45e6-ad02-fec541fe49e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b412e8ba93f4308a4643e7d159e1e01)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:47.361225 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c2d77a-83cf-45e6-ad02-fec541fe49e5 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:47.458601 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-51c2d77a-83cf-45e6-ad02-fec541fe49e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b412e8ba93f4308a4643e7d159e1e01', 'request_id': 'req-51c2d77a-83cf-45e6-ad02-fec541fe49e5'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:47.471760 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-51c2d77a-83cf-45e6-ad02-fec541fe49e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b412e8ba93f4308a4643e7d159e1e01', 'request_id': 'req-51c2d77a-83cf-45e6-ad02-fec541fe49e5'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:47.479977 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-51c2d77a-83cf-45e6-ad02-fec541fe49e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b412e8ba93f4308a4643e7d159e1e01', 'request_id': 'req-51c2d77a-83cf-45e6-ad02-fec541fe49e5'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:47.512628 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-51c2d77a-83cf-45e6-ad02-fec541fe49e5 admin admin] The token cache is being invalidate because project 4b412e8ba93f4308a4643e7d159e1e01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:42:47.513468 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-51c2d77a-83cf-45e6-ad02-fec541fe49e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:47.514433 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-51c2d77a-83cf-45e6-ad02-fec541fe49e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:47.521373 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b131e3-e22f-4691-8ea0-226b88af2bdf service neutron] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:47.524638 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 920/1825] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:42:47 2022] DELETE /identity/v3/projects/4b412e8ba93f4308a4643e7d159e1e01 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:47.544124 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b131e3-e22f-4691-8ea0-226b88af2bdf service neutron] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:47.637582 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d110ac-48f6-496d-a4b6-40d9cb97c13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:47.638228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d110ac-48f6-496d-a4b6-40d9cb97c13f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:47.638507 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d110ac-48f6-496d-a4b6-40d9cb97c13f admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:47.638778 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d110ac-48f6-496d-a4b6-40d9cb97c13f admin admin] PATH_INFO: `/v3/users/54743e5571fe487c8f815461cb699c06` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:47.640795 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d110ac-48f6-496d-a4b6-40d9cb97c13f admin admin] RBAC: Authorizing `identity:delete_user(user_id=54743e5571fe487c8f815461cb699c06)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:47.642744 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d110ac-48f6-496d-a4b6-40d9cb97c13f admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:47.804595 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-41d110ac-48f6-496d-a4b6-40d9cb97c13f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54743e5571fe487c8f815461cb699c06'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:47.805392 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-41d110ac-48f6-496d-a4b6-40d9cb97c13f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54743e5571fe487c8f815461cb699c06'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:47.837832 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-41d110ac-48f6-496d-a4b6-40d9cb97c13f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54743e5571fe487c8f815461cb699c06'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:47.840945 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:47.841801 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c196b4b9-4b2d-47f1-b4f4-1b505539202b None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:47.842089 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c196b4b9-4b2d-47f1-b4f4-1b505539202b None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:47.842454 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c196b4b9-4b2d-47f1-b4f4-1b505539202b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:47.897126 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-41d110ac-48f6-496d-a4b6-40d9cb97c13f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54743e5571fe487c8f815461cb699c06'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:47.897275 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-c196b4b9-4b2d-47f1-b4f4-1b505539202b None None] MFA Rules not processed for user `9e14e86e085d4987aece69fe64867ce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:47.898631 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-c196b4b9-4b2d-47f1-b4f4-1b505539202b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:47.915464 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-41d110ac-48f6-496d-a4b6-40d9cb97c13f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54743e5571fe487c8f815461cb699c06'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:47.928086 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 906/1826] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:42:47 2022] DELETE /identity/v3/users/54743e5571fe487c8f815461cb699c06 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:48.112661 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 921/1827] 10.222.0.58 () {58 vars in 1015 bytes} [Tue Aug 30 14:42:47 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:42:48.225824 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d110ac-48f6-496d-a4b6-40d9cb97c13f admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:48.227396 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-41d110ac-48f6-496d-a4b6-40d9cb97c13f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:48.237647 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c196b4b9-4b2d-47f1-b4f4-1b505539202b None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:48.239321 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-c196b4b9-4b2d-47f1-b4f4-1b505539202b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:48.271872 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c196b4b9-4b2d-47f1-b4f4-1b505539202b None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:48.418788 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:48.439815 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d110ac-48f6-496d-a4b6-40d9cb97c13f admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:48.451306 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2b0295-59ba-4766-88a7-d03cce70e82e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:48.452539 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b0295-59ba-4766-88a7-d03cce70e82e service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:48.452817 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b0295-59ba-4766-88a7-d03cce70e82e service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:48.453087 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b0295-59ba-4766-88a7-d03cce70e82e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:48.462639 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99508e2-4889-48de-b22b-2c3dda05bb4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:48.463308 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99508e2-4889-48de-b22b-2c3dda05bb4d service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:48.463596 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99508e2-4889-48de-b22b-2c3dda05bb4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:48.463868 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99508e2-4889-48de-b22b-2c3dda05bb4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:48.469091 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2b0295-59ba-4766-88a7-d03cce70e82e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:48.470566 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2b0295-59ba-4766-88a7-d03cce70e82e service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:48.491635 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 907/1828] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:48.496689 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99508e2-4889-48de-b22b-2c3dda05bb4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:48.501208 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99508e2-4889-48de-b22b-2c3dda05bb4d service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:48.519543 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 922/1829] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:49.464642 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2b0295-59ba-4766-88a7-d03cce70e82e service neutron] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:49.466425 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-fd2b0295-59ba-4766-88a7-d03cce70e82e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:49.526871 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99508e2-4889-48de-b22b-2c3dda05bb4d service neutron] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:49.534037 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-f99508e2-4889-48de-b22b-2c3dda05bb4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:49.708538 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2b0295-59ba-4766-88a7-d03cce70e82e service neutron] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:49.734279 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99508e2-4889-48de-b22b-2c3dda05bb4d service neutron] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:49.741711 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba6fb38-339a-4b16-8df0-13fbc6e50e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:49.744898 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6fb38-339a-4b16-8df0-13fbc6e50e46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:49.746016 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c8e2ba-56a4-4376-b665-0a491161ae77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:49.746696 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6fb38-339a-4b16-8df0-13fbc6e50e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:49.746936 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6fb38-339a-4b16-8df0-13fbc6e50e46 admin admin] PATH_INFO: `/v3/projects/db900377140041a2a07240729b467475` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:49.747053 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8e2ba-56a4-4376-b665-0a491161ae77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:49.747272 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8e2ba-56a4-4376-b665-0a491161ae77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:49.747557 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8e2ba-56a4-4376-b665-0a491161ae77 admin admin] PATH_INFO: `/v3/users/27348fdcc1834d99969b5fedc54a9fa9` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:49.750818 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c8e2ba-56a4-4376-b665-0a491161ae77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27348fdcc1834d99969b5fedc54a9fa9)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:49.754958 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba6fb38-339a-4b16-8df0-13fbc6e50e46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db900377140041a2a07240729b467475)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:49.755987 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c8e2ba-56a4-4376-b665-0a491161ae77 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:49.767582 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba6fb38-339a-4b16-8df0-13fbc6e50e46 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:49.942548 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-01c8e2ba-56a4-4376-b665-0a491161ae77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27348fdcc1834d99969b5fedc54a9fa9'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:49.943311 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-1ba6fb38-339a-4b16-8df0-13fbc6e50e46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db900377140041a2a07240729b467475', 'request_id': 'req-1ba6fb38-339a-4b16-8df0-13fbc6e50e46'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:49.945030 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-01c8e2ba-56a4-4376-b665-0a491161ae77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27348fdcc1834d99969b5fedc54a9fa9'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:49.963248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-1ba6fb38-339a-4b16-8df0-13fbc6e50e46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db900377140041a2a07240729b467475', 'request_id': 'req-1ba6fb38-339a-4b16-8df0-13fbc6e50e46'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:49.966412 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-01c8e2ba-56a4-4376-b665-0a491161ae77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27348fdcc1834d99969b5fedc54a9fa9'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:49.975885 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-1ba6fb38-339a-4b16-8df0-13fbc6e50e46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db900377140041a2a07240729b467475', 'request_id': 'req-1ba6fb38-339a-4b16-8df0-13fbc6e50e46'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:49.980402 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-01c8e2ba-56a4-4376-b665-0a491161ae77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27348fdcc1834d99969b5fedc54a9fa9'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:49.981812 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-01c8e2ba-56a4-4376-b665-0a491161ae77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27348fdcc1834d99969b5fedc54a9fa9'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:50.003311 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-1ba6fb38-339a-4b16-8df0-13fbc6e50e46 admin admin] The token cache is being invalidate because project db900377140041a2a07240729b467475 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:42:50.003740 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-1ba6fb38-339a-4b16-8df0-13fbc6e50e46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:50.006203 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-1ba6fb38-339a-4b16-8df0-13fbc6e50e46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:50.021859 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 908/1830] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:42:49 2022] DELETE /identity/v3/projects/db900377140041a2a07240729b467475 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:50.022808 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba6fb38-339a-4b16-8df0-13fbc6e50e46 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:50.025982 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-1ba6fb38-339a-4b16-8df0-13fbc6e50e46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:50.071455 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 923/1831] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:42:49 2022] DELETE /identity/v3/users/27348fdcc1834d99969b5fedc54a9fa9 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:50.074263 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0803c8c9-0dd5-4200-b307-0a3d0e708767 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:50.075317 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0803c8c9-0dd5-4200-b307-0a3d0e708767 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:50.078252 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0803c8c9-0dd5-4200-b307-0a3d0e708767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:50.089958 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:50.151480 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-0803c8c9-0dd5-4200-b307-0a3d0e708767 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:50.158562 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-0803c8c9-0dd5-4200-b307-0a3d0e708767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:50.257191 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba6fb38-339a-4b16-8df0-13fbc6e50e46 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:50.413807 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:50.460990 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 924/1832] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:50.461931 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54346ef-ff6e-47bf-b931-98a009aed68a None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:50.462220 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54346ef-ff6e-47bf-b931-98a009aed68a None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:50.464658 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54346ef-ff6e-47bf-b931-98a009aed68a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:50.481195 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4815c1-8ec2-4a0b-9f68-b6786d61660c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:50.481850 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4815c1-8ec2-4a0b-9f68-b6786d61660c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:50.482439 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4815c1-8ec2-4a0b-9f68-b6786d61660c admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:50.483428 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4815c1-8ec2-4a0b-9f68-b6786d61660c admin admin] PATH_INFO: `/v3/users/128cc020d9454b798e9a2ccfe1bff86c` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:50.491482 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4815c1-8ec2-4a0b-9f68-b6786d61660c admin admin] RBAC: Authorizing `identity:delete_user(user_id=128cc020d9454b798e9a2ccfe1bff86c)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:50.493698 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4815c1-8ec2-4a0b-9f68-b6786d61660c admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:50.559769 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-e54346ef-ff6e-47bf-b931-98a009aed68a None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:50.565000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-e54346ef-ff6e-47bf-b931-98a009aed68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:50.586408 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 925/1833] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:50.588628 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce06ab3-15ee-4f0f-89bc-e8c65a3be506 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:50.591346 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce06ab3-15ee-4f0f-89bc-e8c65a3be506 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:50.591760 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-7e4815c1-8ec2-4a0b-9f68-b6786d61660c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '128cc020d9454b798e9a2ccfe1bff86c'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:50.592035 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce06ab3-15ee-4f0f-89bc-e8c65a3be506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:50.594415 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-7e4815c1-8ec2-4a0b-9f68-b6786d61660c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '128cc020d9454b798e9a2ccfe1bff86c'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:50.620153 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-7e4815c1-8ec2-4a0b-9f68-b6786d61660c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '128cc020d9454b798e9a2ccfe1bff86c'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:50.632463 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-7e4815c1-8ec2-4a0b-9f68-b6786d61660c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '128cc020d9454b798e9a2ccfe1bff86c'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:50.650149 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-7e4815c1-8ec2-4a0b-9f68-b6786d61660c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '128cc020d9454b798e9a2ccfe1bff86c'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:50.668609 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-5ce06ab3-15ee-4f0f-89bc-e8c65a3be506 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:50.672483 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-5ce06ab3-15ee-4f0f-89bc-e8c65a3be506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:50.674276 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 909/1834] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:42:50 2022] DELETE /identity/v3/users/128cc020d9454b798e9a2ccfe1bff86c => generated 0 bytes in 652 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:50.679536 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107b895-b0b9-48fa-a702-c776ab19a550 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:50.679960 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107b895-b0b9-48fa-a702-c776ab19a550 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:50.680299 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107b895-b0b9-48fa-a702-c776ab19a550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:50.761189 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-f107b895-b0b9-48fa-a702-c776ab19a550 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:50.763432 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-f107b895-b0b9-48fa-a702-c776ab19a550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:50.895264 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:50.896514 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 926/1835] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:50.897576 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce06ab3-15ee-4f0f-89bc-e8c65a3be506 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:50.899154 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-5ce06ab3-15ee-4f0f-89bc-e8c65a3be506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:50.978109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 910/1836] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:50.982294 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f107b895-b0b9-48fa-a702-c776ab19a550 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:51.028323 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f107b895-b0b9-48fa-a702-c776ab19a550 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:51.058332 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b375780-ad07-4beb-ae0b-d6d22b252964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:51.060556 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b375780-ad07-4beb-ae0b-d6d22b252964 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:51.061786 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b375780-ad07-4beb-ae0b-d6d22b252964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:51.063965 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b375780-ad07-4beb-ae0b-d6d22b252964 admin admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:51.064955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b375780-ad07-4beb-ae0b-d6d22b252964 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:42:51.066962 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce06ab3-15ee-4f0f-89bc-e8c65a3be506 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:51.067398 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b375780-ad07-4beb-ae0b-d6d22b252964 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:51.069282 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b375780-ad07-4beb-ae0b-d6d22b252964 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:51.078442 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f010ac-c9a4-4565-8ea1-972ddca5f34d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:51.078988 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f010ac-c9a4-4565-8ea1-972ddca5f34d service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:51.079293 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f010ac-c9a4-4565-8ea1-972ddca5f34d service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:51.079567 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f010ac-c9a4-4565-8ea1-972ddca5f34d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:51.082557 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-50f010ac-c9a4-4565-8ea1-972ddca5f34d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:51.111094 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 911/1837] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 14:42:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:42:51.112107 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b375780-ad07-4beb-ae0b-d6d22b252964 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:51.118638 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f010ac-c9a4-4565-8ea1-972ddca5f34d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:51.120139 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f010ac-c9a4-4565-8ea1-972ddca5f34d service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:51.125453 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b375780-ad07-4beb-ae0b-d6d22b252964 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:51.146916 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f775e01-3458-4a78-9e8a-ad9b074c2df3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:51.147230 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 927/1838] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:51.147874 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f775e01-3458-4a78-9e8a-ad9b074c2df3 service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:51.148235 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f775e01-3458-4a78-9e8a-ad9b074c2df3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:51.148616 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f775e01-3458-4a78-9e8a-ad9b074c2df3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:51.149081 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f010ac-c9a4-4565-8ea1-972ddca5f34d service neutron] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:51.172075 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f010ac-c9a4-4565-8ea1-972ddca5f34d service neutron] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:51.180906 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f775e01-3458-4a78-9e8a-ad9b074c2df3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:51.182678 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f775e01-3458-4a78-9e8a-ad9b074c2df3 service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:51.184273 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196e7a48-fae1-4009-85fe-52d891d072f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:51.184900 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e7a48-fae1-4009-85fe-52d891d072f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:51.185232 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e7a48-fae1-4009-85fe-52d891d072f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:51.185507 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e7a48-fae1-4009-85fe-52d891d072f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:51.186444 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196e7a48-fae1-4009-85fe-52d891d072f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:51.193428 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196e7a48-fae1-4009-85fe-52d891d072f6 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:51.231599 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 912/1839] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:51.261952 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 928/1840] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:42:51 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:51.266553 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f775e01-3458-4a78-9e8a-ad9b074c2df3 service neutron] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:51.287648 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f775e01-3458-4a78-9e8a-ad9b074c2df3 service neutron] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:51.309864 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f3aa9e-b46f-4871-84f9-26e7bc1b508b None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:51.310435 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f3aa9e-b46f-4871-84f9-26e7bc1b508b None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:51.312015 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f3aa9e-b46f-4871-84f9-26e7bc1b508b None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:51.313810 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 929/1841] 10.222.0.30 () {54 vars in 921 bytes} [Tue Aug 30 14:42:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:42:51.470192 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd520b-029b-4155-82ec-3614b6e96ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:51.470839 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd520b-029b-4155-82ec-3614b6e96ee6 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:51.471126 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd520b-029b-4155-82ec-3614b6e96ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:51.471434 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd520b-029b-4155-82ec-3614b6e96ee6 admin admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:51.472779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbd520b-029b-4155-82ec-3614b6e96ee6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:51.474754 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbd520b-029b-4155-82ec-3614b6e96ee6 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:51.517177 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73947e4-a3b4-43a9-8a05-9a3b47304ed7 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:51.517478 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73947e4-a3b4-43a9-8a05-9a3b47304ed7 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:51.519087 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73947e4-a3b4-43a9-8a05-9a3b47304ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:51.549641 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 913/1842] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:42:51 2022] POST /identity/v3/users => generated 293 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:51.553214 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd520b-029b-4155-82ec-3614b6e96ee6 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:51.600574 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-c73947e4-a3b4-43a9-8a05-9a3b47304ed7 None None] MFA Rules not processed for user `8878de66a03044e38e507028853a0cd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:51.610643 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-c73947e4-a3b4-43a9-8a05-9a3b47304ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:51.620306 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd520b-029b-4155-82ec-3614b6e96ee6 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:51.669853 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4303a3-efba-4165-b731-4fc54ad19967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:51.670555 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4303a3-efba-4165-b731-4fc54ad19967 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:51.670841 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4303a3-efba-4165-b731-4fc54ad19967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:51.671115 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4303a3-efba-4165-b731-4fc54ad19967 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:51.671811 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4303a3-efba-4165-b731-4fc54ad19967 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:51.690547 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4303a3-efba-4165-b731-4fc54ad19967 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:51.745330 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 914/1843] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:51 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:51.748913 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4303a3-efba-4165-b731-4fc54ad19967 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:51.763122 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 930/1844] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Aug 30 14:42:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:51.785645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4303a3-efba-4165-b731-4fc54ad19967 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:51.816136 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cc4e51-aa43-49eb-a825-0cc6754aa04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:51.816772 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc4e51-aa43-49eb-a825-0cc6754aa04f admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:51.817055 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc4e51-aa43-49eb-a825-0cc6754aa04f admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:51.817332 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cc4e51-aa43-49eb-a825-0cc6754aa04f admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:51.817985 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cc4e51-aa43-49eb-a825-0cc6754aa04f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:51.834136 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cc4e51-aa43-49eb-a825-0cc6754aa04f admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:51.872511 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 915/1845] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:51 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:51.880545 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73947e4-a3b4-43a9-8a05-9a3b47304ed7 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:51.896427 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73947e4-a3b4-43a9-8a05-9a3b47304ed7 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:51.919864 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b29f746-3a49-4a56-bbe3-a081c7364c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:51.920808 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29f746-3a49-4a56-bbe3-a081c7364c30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:51.921255 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29f746-3a49-4a56-bbe3-a081c7364c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:51.921821 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29f746-3a49-4a56-bbe3-a081c7364c30 admin admin] PATH_INFO: `/v3/projects/c4880227055646de88afca1aa55a2f90/users/eec36a9fb3194b2092603625dce5ab88/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:51.926006 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:51.965406 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b29f746-3a49-4a56-bbe3-a081c7364c30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4880227055646de88afca1aa55a2f90, user_id=eec36a9fb3194b2092603625dce5ab88, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:51.968984 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b29f746-3a49-4a56-bbe3-a081c7364c30 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:52.023239 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 931/1846] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:42:51 2022] PUT /identity/v3/projects/c4880227055646de88afca1aa55a2f90/users/eec36a9fb3194b2092603625dce5ab88/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:52.031501 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1afd75-5e24-48d3-aac3-927d1e457da8 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:52.031820 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1afd75-5e24-48d3-aac3-927d1e457da8 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:52.033143 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1afd75-5e24-48d3-aac3-927d1e457da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:52.092512 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-9c1afd75-5e24-48d3-aac3-927d1e457da8 None None] MFA Rules not processed for user `eec36a9fb3194b2092603625dce5ab88`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:52.093836 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-9c1afd75-5e24-48d3-aac3-927d1e457da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:52.216603 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b29f746-3a49-4a56-bbe3-a081c7364c30 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:52.218678 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-5b29f746-3a49-4a56-bbe3-a081c7364c30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:52.260938 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 916/1847] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:52.266935 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1afd75-5e24-48d3-aac3-927d1e457da8 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:52.365277 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:52.501220 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b29f746-3a49-4a56-bbe3-a081c7364c30 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:52.516304 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1afd75-5e24-48d3-aac3-927d1e457da8 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:52.528145 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacf91f6-d833-4d1f-aef2-2d4b316a6fb7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:52.530519 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf91f6-d833-4d1f-aef2-2d4b316a6fb7 service nova] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:52.531129 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd508b09-5c78-42d7-b12f-07cf821e15c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:52.531816 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd508b09-5c78-42d7-b12f-07cf821e15c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:52.532098 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd508b09-5c78-42d7-b12f-07cf821e15c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:52.532376 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd508b09-5c78-42d7-b12f-07cf821e15c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:52.532940 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf91f6-d833-4d1f-aef2-2d4b316a6fb7 service nova] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:52.533061 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd508b09-5c78-42d7-b12f-07cf821e15c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:52.534901 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf91f6-d833-4d1f-aef2-2d4b316a6fb7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:52.535037 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd508b09-5c78-42d7-b12f-07cf821e15c6 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:52.570701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacf91f6-d833-4d1f-aef2-2d4b316a6fb7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:52.576721 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacf91f6-d833-4d1f-aef2-2d4b316a6fb7 service nova] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:52.611268 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 917/1848] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:42:52 2022] POST /identity/v3/projects => generated 383 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:52.611835 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 932/1849] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:52.622332 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd508b09-5c78-42d7-b12f-07cf821e15c6 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:52.623337 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacf91f6-d833-4d1f-aef2-2d4b316a6fb7 service nova] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:52.630913 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-cd508b09-5c78-42d7-b12f-07cf821e15c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:52.650267 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacf91f6-d833-4d1f-aef2-2d4b316a6fb7 service nova] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:52.870378 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd508b09-5c78-42d7-b12f-07cf821e15c6 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:52.889664 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf83c922-4c02-40ac-8eaa-d959efe85372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:52.890346 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf83c922-4c02-40ac-8eaa-d959efe85372 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:52.890567 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52432397-5c6a-47c1-94a2-cb2d604c61ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:52.890682 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf83c922-4c02-40ac-8eaa-d959efe85372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:52.890891 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf83c922-4c02-40ac-8eaa-d959efe85372 admin admin] PATH_INFO: `/v3/projects/82470c9b7483440f9b87d7c034fccfba` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:52.891138 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52432397-5c6a-47c1-94a2-cb2d604c61ab admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:52.897608 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf83c922-4c02-40ac-8eaa-d959efe85372 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82470c9b7483440f9b87d7c034fccfba)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:52.919667 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52432397-5c6a-47c1-94a2-cb2d604c61ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:52.919987 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52432397-5c6a-47c1-94a2-cb2d604c61ab admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:52.921342 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52432397-5c6a-47c1-94a2-cb2d604c61ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:52.923176 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf83c922-4c02-40ac-8eaa-d959efe85372 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:52.923448 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52432397-5c6a-47c1-94a2-cb2d604c61ab admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:52.960973 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 933/1850] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:42:52 2022] POST /identity/v3/users => generated 292 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:52.962266 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de661acc-898f-42e6-acac-c1ed314b88fc None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:52.962548 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de661acc-898f-42e6-acac-c1ed314b88fc None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:52.962823 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de661acc-898f-42e6-acac-c1ed314b88fc None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:52.975577 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 934/1851] 10.222.0.30 () {54 vars in 921 bytes} [Tue Aug 30 14:42:52 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:42:52.976552 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de661acc-898f-42e6-acac-c1ed314b88fc None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:53.000334 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de661acc-898f-42e6-acac-c1ed314b88fc None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:53.011995 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a98bbb-5b3d-4977-a4db-5dd0dc09e009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:53.012628 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a98bbb-5b3d-4977-a4db-5dd0dc09e009 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:53.012911 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a98bbb-5b3d-4977-a4db-5dd0dc09e009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:53.013183 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a98bbb-5b3d-4977-a4db-5dd0dc09e009 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:53.013835 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a98bbb-5b3d-4977-a4db-5dd0dc09e009 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:53.015975 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a98bbb-5b3d-4977-a4db-5dd0dc09e009 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:53.047632 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 935/1852] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:52 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:53.048944 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b9599-9a3f-459e-a0e3-f91e07102fd5 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:53.049232 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b9599-9a3f-459e-a0e3-f91e07102fd5 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:53.049507 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b9599-9a3f-459e-a0e3-f91e07102fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:53.087851 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-cf83c922-4c02-40ac-8eaa-d959efe85372 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82470c9b7483440f9b87d7c034fccfba', 'request_id': 'req-cf83c922-4c02-40ac-8eaa-d959efe85372'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:53.095614 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-cf83c922-4c02-40ac-8eaa-d959efe85372 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82470c9b7483440f9b87d7c034fccfba', 'request_id': 'req-cf83c922-4c02-40ac-8eaa-d959efe85372'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:53.110354 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-cf83c922-4c02-40ac-8eaa-d959efe85372 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82470c9b7483440f9b87d7c034fccfba', 'request_id': 'req-cf83c922-4c02-40ac-8eaa-d959efe85372'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:53.117766 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-288b9599-9a3f-459e-a0e3-f91e07102fd5 None None] MFA Rules not processed for user `8878de66a03044e38e507028853a0cd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:53.119029 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-288b9599-9a3f-459e-a0e3-f91e07102fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:53.132597 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-cf83c922-4c02-40ac-8eaa-d959efe85372 admin admin] The token cache is being invalidate because project 82470c9b7483440f9b87d7c034fccfba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:42:53.133275 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-cf83c922-4c02-40ac-8eaa-d959efe85372 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:53.134295 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-cf83c922-4c02-40ac-8eaa-d959efe85372 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:53.136916 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 918/1853] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:42:52 2022] DELETE /identity/v3/projects/82470c9b7483440f9b87d7c034fccfba => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:53.138275 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf83c922-4c02-40ac-8eaa-d959efe85372 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:53.141898 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-cf83c922-4c02-40ac-8eaa-d959efe85372 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:53.195791 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:53.262018 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:53.266923 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf83c922-4c02-40ac-8eaa-d959efe85372 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:53.299559 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b7c7c-a23c-4e62-aa8a-56686995bda8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:53.300208 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b7c7c-a23c-4e62-aa8a-56686995bda8 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:53.300564 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b7c7c-a23c-4e62-aa8a-56686995bda8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:53.300846 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b7c7c-a23c-4e62-aa8a-56686995bda8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:53.301513 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542b7c7c-a23c-4e62-aa8a-56686995bda8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:53.303338 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542b7c7c-a23c-4e62-aa8a-56686995bda8 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:53.318185 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 919/1854] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:53 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:53.331283 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b7c7c-a23c-4e62-aa8a-56686995bda8 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:53.343446 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-542b7c7c-a23c-4e62-aa8a-56686995bda8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:53.369558 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 936/1855] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Aug 30 14:42:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:53.372840 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288b9599-9a3f-459e-a0e3-f91e07102fd5 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:53.386142 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b7c7c-a23c-4e62-aa8a-56686995bda8 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:53.393306 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173fba9c-4fbc-4b60-86f1-fdd4ea4c726f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:53.394447 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173fba9c-4fbc-4b60-86f1-fdd4ea4c726f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:53.395083 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173fba9c-4fbc-4b60-86f1-fdd4ea4c726f admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:53.396768 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173fba9c-4fbc-4b60-86f1-fdd4ea4c726f admin admin] PATH_INFO: `/v3/users/f4ab48e03d4640e586b51c126e1f9b9a` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:53.399052 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173fba9c-4fbc-4b60-86f1-fdd4ea4c726f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4ab48e03d4640e586b51c126e1f9b9a)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:53.402699 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173fba9c-4fbc-4b60-86f1-fdd4ea4c726f admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:53.413132 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288b9599-9a3f-459e-a0e3-f91e07102fd5 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:53.433706 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0185f6e1-45fc-4319-9d70-40718067926f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:53.435140 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0185f6e1-45fc-4319-9d70-40718067926f admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:53.435790 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0185f6e1-45fc-4319-9d70-40718067926f admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:53.436264 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0185f6e1-45fc-4319-9d70-40718067926f admin admin] PATH_INFO: `/v3/projects/2c07589ecf8e4e3cb4094d129d3bb890/users/d9a194c87e554a858beab1b03709e146/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:53.463100 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0185f6e1-45fc-4319-9d70-40718067926f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c07589ecf8e4e3cb4094d129d3bb890, user_id=d9a194c87e554a858beab1b03709e146, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:53.468489 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0185f6e1-45fc-4319-9d70-40718067926f admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:53.491175 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 937/1856] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:42:53 2022] PUT /identity/v3/projects/2c07589ecf8e4e3cb4094d129d3bb890/users/d9a194c87e554a858beab1b03709e146/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:53.507282 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d751f0b7-325f-42f5-9953-523e324aca3f None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:53.507282 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d751f0b7-325f-42f5-9953-523e324aca3f None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:53.507282 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d751f0b7-325f-42f5-9953-523e324aca3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:53.515679 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-173fba9c-4fbc-4b60-86f1-fdd4ea4c726f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4ab48e03d4640e586b51c126e1f9b9a'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:53.516527 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-173fba9c-4fbc-4b60-86f1-fdd4ea4c726f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4ab48e03d4640e586b51c126e1f9b9a'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:53.543245 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-173fba9c-4fbc-4b60-86f1-fdd4ea4c726f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4ab48e03d4640e586b51c126e1f9b9a'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:53.552639 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-173fba9c-4fbc-4b60-86f1-fdd4ea4c726f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4ab48e03d4640e586b51c126e1f9b9a'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:53.566890 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-d751f0b7-325f-42f5-9953-523e324aca3f None None] MFA Rules not processed for user `d9a194c87e554a858beab1b03709e146`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:53.568378 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-d751f0b7-325f-42f5-9953-523e324aca3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:53.574704 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-173fba9c-4fbc-4b60-86f1-fdd4ea4c726f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4ab48e03d4640e586b51c126e1f9b9a'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:53.617187 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 920/1857] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:42:53 2022] DELETE /identity/v3/users/f4ab48e03d4640e586b51c126e1f9b9a => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:53.704548 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 938/1858] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:53.709311 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173fba9c-4fbc-4b60-86f1-fdd4ea4c726f admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:53.711114 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-173fba9c-4fbc-4b60-86f1-fdd4ea4c726f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:53.743201 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d751f0b7-325f-42f5-9953-523e324aca3f None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:53.745266 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-d751f0b7-325f-42f5-9953-523e324aca3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:53.874585 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173fba9c-4fbc-4b60-86f1-fdd4ea4c726f admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:53.888089 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a42f5-b57b-44e1-82a8-bd94f077bbe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:53.888649 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a42f5-b57b-44e1-82a8-bd94f077bbe4 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:53.888959 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a42f5-b57b-44e1-82a8-bd94f077bbe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:53.889288 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a42f5-b57b-44e1-82a8-bd94f077bbe4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:53.890247 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33a42f5-b57b-44e1-82a8-bd94f077bbe4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:53.892217 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33a42f5-b57b-44e1-82a8-bd94f077bbe4 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:53.902930 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d751f0b7-325f-42f5-9953-523e324aca3f None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:53.917012 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b443688-b08e-4eba-9453-0f6a4b8d2374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:53.917786 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b443688-b08e-4eba-9453-0f6a4b8d2374 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:53.918085 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b443688-b08e-4eba-9453-0f6a4b8d2374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:53.918395 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b443688-b08e-4eba-9453-0f6a4b8d2374 admin admin] PATH_INFO: `/v3/projects/f07273ec04bb4262a5667aa7f3af3a5b` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:53.920019 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b443688-b08e-4eba-9453-0f6a4b8d2374 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f07273ec04bb4262a5667aa7f3af3a5b)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:53.923936 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b443688-b08e-4eba-9453-0f6a4b8d2374 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:53.966048 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 921/1859] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:42:53 2022] POST /identity/v3/projects => generated 383 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:53.966048 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a42f5-b57b-44e1-82a8-bd94f077bbe4 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:53.967839 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-b33a42f5-b57b-44e1-82a8-bd94f077bbe4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:54.002122 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:54.009545 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-1b443688-b08e-4eba-9453-0f6a4b8d2374 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f07273ec04bb4262a5667aa7f3af3a5b', 'request_id': 'req-1b443688-b08e-4eba-9453-0f6a4b8d2374'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:54.021068 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-1b443688-b08e-4eba-9453-0f6a4b8d2374 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f07273ec04bb4262a5667aa7f3af3a5b', 'request_id': 'req-1b443688-b08e-4eba-9453-0f6a4b8d2374'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:54.024475 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:54.033144 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-1b443688-b08e-4eba-9453-0f6a4b8d2374 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f07273ec04bb4262a5667aa7f3af3a5b', 'request_id': 'req-1b443688-b08e-4eba-9453-0f6a4b8d2374'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:54.052730 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-1b443688-b08e-4eba-9453-0f6a4b8d2374 admin admin] The token cache is being invalidate because project f07273ec04bb4262a5667aa7f3af3a5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:42:54.053474 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-1b443688-b08e-4eba-9453-0f6a4b8d2374 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:54.054552 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-1b443688-b08e-4eba-9453-0f6a4b8d2374 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:54.059260 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 939/1860] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:42:53 2022] DELETE /identity/v3/projects/f07273ec04bb4262a5667aa7f3af3a5b => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:42:54.059260 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b443688-b08e-4eba-9453-0f6a4b8d2374 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:54.060203 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-1b443688-b08e-4eba-9453-0f6a4b8d2374 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:54.096409 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a42f5-b57b-44e1-82a8-bd94f077bbe4 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:54.158411 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b443688-b08e-4eba-9453-0f6a4b8d2374 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:54.165964 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a57749a-aa5d-4fcf-b8ab-bd90d4eb7330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:54.166994 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57749a-aa5d-4fcf-b8ab-bd90d4eb7330 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:54.167320 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57749a-aa5d-4fcf-b8ab-bd90d4eb7330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:54.167598 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57749a-aa5d-4fcf-b8ab-bd90d4eb7330 admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:54.168383 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a57749a-aa5d-4fcf-b8ab-bd90d4eb7330 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:54.171628 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a57749a-aa5d-4fcf-b8ab-bd90d4eb7330 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:54.198040 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f3b1ec-2ec2-42c2-8645-343ae1083df7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:54.198896 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3b1ec-2ec2-42c2-8645-343ae1083df7 service nova] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:54.199325 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3b1ec-2ec2-42c2-8645-343ae1083df7 service nova] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:54.199754 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3b1ec-2ec2-42c2-8645-343ae1083df7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:54.202388 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-a5f3b1ec-2ec2-42c2-8645-343ae1083df7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:54.206394 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 940/1861] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:42:54 2022] POST /identity/v3/users => generated 298 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:54.207542 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a57749a-aa5d-4fcf-b8ab-bd90d4eb7330 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:54.209594 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-1a57749a-aa5d-4fcf-b8ab-bd90d4eb7330 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:54.220600 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f3b1ec-2ec2-42c2-8645-343ae1083df7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:54.225852 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f3b1ec-2ec2-42c2-8645-343ae1083df7 service nova] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:54.228478 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a57749a-aa5d-4fcf-b8ab-bd90d4eb7330 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:54.237019 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff76f548-3c9e-42e5-922f-09880f8c4c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:54.238032 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76f548-3c9e-42e5-922f-09880f8c4c7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:54.238419 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76f548-3c9e-42e5-922f-09880f8c4c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:54.238777 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76f548-3c9e-42e5-922f-09880f8c4c7e admin admin] PATH_INFO: `/v3/users/4246de3e7143447d934fb7dde53b3dcc` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:54.240552 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff76f548-3c9e-42e5-922f-09880f8c4c7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4246de3e7143447d934fb7dde53b3dcc)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:54.241003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 922/1862] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:54.242126 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f3b1ec-2ec2-42c2-8645-343ae1083df7 service nova] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:54.255850 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f3b1ec-2ec2-42c2-8645-343ae1083df7 service nova] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:54.258524 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff76f548-3c9e-42e5-922f-09880f8c4c7e admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:54.265454 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b54ab5-288a-4cac-bb0a-68ac55426cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:54.267203 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b54ab5-288a-4cac-bb0a-68ac55426cd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:54.269350 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b54ab5-288a-4cac-bb0a-68ac55426cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:54.269771 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b54ab5-288a-4cac-bb0a-68ac55426cd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:54.271399 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b54ab5-288a-4cac-bb0a-68ac55426cd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:54.277842 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b54ab5-288a-4cac-bb0a-68ac55426cd2 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:54.319935 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 923/1863] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:54 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:54.324324 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b54ab5-288a-4cac-bb0a-68ac55426cd2 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:54.329723 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-ff76f548-3c9e-42e5-922f-09880f8c4c7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4246de3e7143447d934fb7dde53b3dcc'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:54.330545 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-ff76f548-3c9e-42e5-922f-09880f8c4c7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4246de3e7143447d934fb7dde53b3dcc'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:54.338039 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b54ab5-288a-4cac-bb0a-68ac55426cd2 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:54.343000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-ff76f548-3c9e-42e5-922f-09880f8c4c7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4246de3e7143447d934fb7dde53b3dcc'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:54.348863 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a8a61-269c-4cae-a354-2bf164076f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:54.350006 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a8a61-269c-4cae-a354-2bf164076f9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:54.350387 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a8a61-269c-4cae-a354-2bf164076f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:54.350771 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a8a61-269c-4cae-a354-2bf164076f9b admin admin] PATH_INFO: `/v3/projects/c2c79cd6cac141c3892a3131d462c287/users/cbf3579558d54be788f4030e3bc5d053/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:54.354208 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-ff76f548-3c9e-42e5-922f-09880f8c4c7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4246de3e7143447d934fb7dde53b3dcc'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:54.359089 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-ff76f548-3c9e-42e5-922f-09880f8c4c7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4246de3e7143447d934fb7dde53b3dcc'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:54.382665 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79a8a61-269c-4cae-a354-2bf164076f9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2c79cd6cac141c3892a3131d462c287, user_id=cbf3579558d54be788f4030e3bc5d053, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:54.389303 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79a8a61-269c-4cae-a354-2bf164076f9b admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:54.399287 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 941/1864] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:42:54 2022] DELETE /identity/v3/users/4246de3e7143447d934fb7dde53b3dcc => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:54.415183 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 924/1865] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:42:54 2022] PUT /identity/v3/projects/c2c79cd6cac141c3892a3131d462c287/users/cbf3579558d54be788f4030e3bc5d053/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:42:54.419082 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff76f548-3c9e-42e5-922f-09880f8c4c7e admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:54.421400 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-ff76f548-3c9e-42e5-922f-09880f8c4c7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:54.481063 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:DELETE=14,keystone:SELECT=81,keystone:INSERT=5 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:54.514227 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff76f548-3c9e-42e5-922f-09880f8c4c7e admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:54.582855 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e40a659-d366-4c8d-be21-7a4f518b2944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:54.583567 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40a659-d366-4c8d-be21-7a4f518b2944 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:54.583856 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40a659-d366-4c8d-be21-7a4f518b2944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:54.585552 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40a659-d366-4c8d-be21-7a4f518b2944 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:54.585552 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e40a659-d366-4c8d-be21-7a4f518b2944 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:54.586373 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e40a659-d366-4c8d-be21-7a4f518b2944 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:54.615277 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 942/1866] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:54 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:54.619502 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a8a61-269c-4cae-a354-2bf164076f9b admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:54.652421 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a8a61-269c-4cae-a354-2bf164076f9b admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:54.666708 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd91265-aaf3-4ab0-8284-649cd5d64508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:54.667424 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd91265-aaf3-4ab0-8284-649cd5d64508 admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:54.667710 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd91265-aaf3-4ab0-8284-649cd5d64508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:54.670151 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd91265-aaf3-4ab0-8284-649cd5d64508 admin admin] PATH_INFO: `/v3/projects/c2c79cd6cac141c3892a3131d462c287/users/cbf3579558d54be788f4030e3bc5d053/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:54.675346 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd91265-aaf3-4ab0-8284-649cd5d64508 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2c79cd6cac141c3892a3131d462c287, user_id=cbf3579558d54be788f4030e3bc5d053, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:54.682236 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd91265-aaf3-4ab0-8284-649cd5d64508 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:54.712911 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 925/1867] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:42:54 2022] PUT /identity/v3/projects/c2c79cd6cac141c3892a3131d462c287/users/cbf3579558d54be788f4030e3bc5d053/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:54.723971 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67e257-1e10-454c-97df-5083826b1569 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:54.724397 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67e257-1e10-454c-97df-5083826b1569 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:54.724596 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67e257-1e10-454c-97df-5083826b1569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:54.769865 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-8e67e257-1e10-454c-97df-5083826b1569 None None] MFA Rules not processed for user `cbf3579558d54be788f4030e3bc5d053`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:54.771117 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-8e67e257-1e10-454c-97df-5083826b1569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:54.937915 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 943/1868] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:54 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:54.951011 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd91265-aaf3-4ab0-8284-649cd5d64508 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:54.955939 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-0cd91265-aaf3-4ab0-8284-649cd5d64508 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:55.095658 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd91265-aaf3-4ab0-8284-649cd5d64508 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:55.101156 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d3ecc9-1e39-4bac-b8a0-12507a44cbaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:55.101996 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d3ecc9-1e39-4bac-b8a0-12507a44cbaf service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:55.102464 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d3ecc9-1e39-4bac-b8a0-12507a44cbaf service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:55.102920 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d3ecc9-1e39-4bac-b8a0-12507a44cbaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:55.105228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-06d3ecc9-1e39-4bac-b8a0-12507a44cbaf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:55.114417 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67e257-1e10-454c-97df-5083826b1569 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:55.116155 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-8e67e257-1e10-454c-97df-5083826b1569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:55.141934 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:55.287308 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67e257-1e10-454c-97df-5083826b1569 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:55.292448 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d3ecc9-1e39-4bac-b8a0-12507a44cbaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:55.293028 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ce57a5-db1d-4549-921c-e529cd36f8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:55.293684 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce57a5-db1d-4549-921c-e529cd36f8ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:55.293965 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce57a5-db1d-4549-921c-e529cd36f8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:55.294241 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce57a5-db1d-4549-921c-e529cd36f8ee admin admin] PATH_INFO: `/v3/projects/768a2489be74473b9cf346bdf346948e` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:55.294356 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d3ecc9-1e39-4bac-b8a0-12507a44cbaf service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:55.295992 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ce57a5-db1d-4549-921c-e529cd36f8ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=768a2489be74473b9cf346bdf346948e)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:55.297938 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ce57a5-db1d-4549-921c-e529cd36f8ee admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:55.306757 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 926/1869] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:55.307103 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:55.323783 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d3ecc9-1e39-4bac-b8a0-12507a44cbaf service cinder] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:55.328008 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-06d3ecc9-1e39-4bac-b8a0-12507a44cbaf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:55.378778 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d3ecc9-1e39-4bac-b8a0-12507a44cbaf service cinder] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:55.391540 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-b2ce57a5-db1d-4549-921c-e529cd36f8ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '768a2489be74473b9cf346bdf346948e', 'request_id': 'req-b2ce57a5-db1d-4549-921c-e529cd36f8ee'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:55.406522 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-b2ce57a5-db1d-4549-921c-e529cd36f8ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '768a2489be74473b9cf346bdf346948e', 'request_id': 'req-b2ce57a5-db1d-4549-921c-e529cd36f8ee'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:55.428733 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-b2ce57a5-db1d-4549-921c-e529cd36f8ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '768a2489be74473b9cf346bdf346948e', 'request_id': 'req-b2ce57a5-db1d-4549-921c-e529cd36f8ee'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:55.461516 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-b2ce57a5-db1d-4549-921c-e529cd36f8ee admin admin] The token cache is being invalidate because project 768a2489be74473b9cf346bdf346948e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:42:55.461824 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-b2ce57a5-db1d-4549-921c-e529cd36f8ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:55.462799 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-b2ce57a5-db1d-4549-921c-e529cd36f8ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:55.466164 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 944/1870] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:42:55 2022] DELETE /identity/v3/projects/768a2489be74473b9cf346bdf346948e => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:55.475269 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017d86e-9a77-407c-a44f-c0d15d886990 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:55.475547 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017d86e-9a77-407c-a44f-c0d15d886990 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:55.475824 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017d86e-9a77-407c-a44f-c0d15d886990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:55.542398 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-2017d86e-9a77-407c-a44f-c0d15d886990 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:55.543912 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-2017d86e-9a77-407c-a44f-c0d15d886990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:55.576584 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc46c0-978a-4d6e-9e11-83e8f3895bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:55.577841 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc46c0-978a-4d6e-9e11-83e8f3895bdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:55.578490 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc46c0-978a-4d6e-9e11-83e8f3895bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:55.578958 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc46c0-978a-4d6e-9e11-83e8f3895bdb admin admin] PATH_INFO: `/v3/projects/3ddeeff8356d4ab08a6260ca7750343c` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:55.581192 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbc46c0-978a-4d6e-9e11-83e8f3895bdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ddeeff8356d4ab08a6260ca7750343c)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:55.583465 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbc46c0-978a-4d6e-9e11-83e8f3895bdb admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:55.641763 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 945/1871] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:55.650053 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44414228-5cd6-414b-ade3-e59d33f02aad None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:55.650561 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44414228-5cd6-414b-ade3-e59d33f02aad None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:55.651170 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44414228-5cd6-414b-ade3-e59d33f02aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:55.671664 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-ddbc46c0-978a-4d6e-9e11-83e8f3895bdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ddeeff8356d4ab08a6260ca7750343c', 'request_id': 'req-ddbc46c0-978a-4d6e-9e11-83e8f3895bdb'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:55.684270 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-ddbc46c0-978a-4d6e-9e11-83e8f3895bdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ddeeff8356d4ab08a6260ca7750343c', 'request_id': 'req-ddbc46c0-978a-4d6e-9e11-83e8f3895bdb'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:55.688934 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-44414228-5cd6-414b-ade3-e59d33f02aad None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:55.690966 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-44414228-5cd6-414b-ade3-e59d33f02aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:55.700200 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-ddbc46c0-978a-4d6e-9e11-83e8f3895bdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ddeeff8356d4ab08a6260ca7750343c', 'request_id': 'req-ddbc46c0-978a-4d6e-9e11-83e8f3895bdb'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:55.722688 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-ddbc46c0-978a-4d6e-9e11-83e8f3895bdb admin admin] The token cache is being invalidate because project 3ddeeff8356d4ab08a6260ca7750343c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:42:55.724959 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-ddbc46c0-978a-4d6e-9e11-83e8f3895bdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:55.726326 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-ddbc46c0-978a-4d6e-9e11-83e8f3895bdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:42:55.730100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 927/1872] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:42:55 2022] DELETE /identity/v3/projects/3ddeeff8356d4ab08a6260ca7750343c => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:55.738899 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5720d995-636a-4d11-b5a0-62862664919b None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:55.739891 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5720d995-636a-4d11-b5a0-62862664919b None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:55.740257 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5720d995-636a-4d11-b5a0-62862664919b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:55.781556 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-5720d995-636a-4d11-b5a0-62862664919b None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:55.782940 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-5720d995-636a-4d11-b5a0-62862664919b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:55.790866 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 946/1873] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:55.794877 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44414228-5cd6-414b-ade3-e59d33f02aad None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:55.803254 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 928/1874] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:55.811903 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d7709-4dc1-47ef-9482-e2b48b12d34e None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:55.812545 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d7709-4dc1-47ef-9482-e2b48b12d34e None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:55.813093 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d7709-4dc1-47ef-9482-e2b48b12d34e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:55.834252 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44414228-5cd6-414b-ade3-e59d33f02aad None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:55.842935 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96fde9f-fc05-4916-b9e1-ec689d449f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:55.843854 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96fde9f-fc05-4916-b9e1-ec689d449f1d admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:55.844268 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96fde9f-fc05-4916-b9e1-ec689d449f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:55.844618 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96fde9f-fc05-4916-b9e1-ec689d449f1d admin admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:55.845381 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-0d1d7709-4dc1-47ef-9482-e2b48b12d34e None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:55.845495 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96fde9f-fc05-4916-b9e1-ec689d449f1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:42:55.845850 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96fde9f-fc05-4916-b9e1-ec689d449f1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:55.846718 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-0d1d7709-4dc1-47ef-9482-e2b48b12d34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:55.848491 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96fde9f-fc05-4916-b9e1-ec689d449f1d admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:55.868063 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 947/1875] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 14:42:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:42:55.868615 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 929/1876] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:55.872241 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96fde9f-fc05-4916-b9e1-ec689d449f1d admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:55.874600 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1d7709-4dc1-47ef-9482-e2b48b12d34e None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:55.885751 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1d7709-4dc1-47ef-9482-e2b48b12d34e None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:55.900793 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ee9c62-f42b-4e81-b6ce-ab6272d86e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:55.901569 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ee9c62-f42b-4e81-b6ce-ab6272d86e7a admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:55.902006 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ee9c62-f42b-4e81-b6ce-ab6272d86e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:55.902381 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ee9c62-f42b-4e81-b6ce-ab6272d86e7a admin admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:55.903147 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ee9c62-f42b-4e81-b6ce-ab6272d86e7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:42:55.903492 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ee9c62-f42b-4e81-b6ce-ab6272d86e7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:55.905425 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ee9c62-f42b-4e81-b6ce-ab6272d86e7a admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:55.910829 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96fde9f-fc05-4916-b9e1-ec689d449f1d admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:55.936885 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34708deb-c37f-4772-93e8-e52265db3493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:55.939118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34708deb-c37f-4772-93e8-e52265db3493 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:55.939377 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 930/1877] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 14:42:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:42:55.939915 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34708deb-c37f-4772-93e8-e52265db3493 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:55.940933 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34708deb-c37f-4772-93e8-e52265db3493 admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:55.942625 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34708deb-c37f-4772-93e8-e52265db3493 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:55.943354 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ee9c62-f42b-4e81-b6ce-ab6272d86e7a admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:55.948683 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34708deb-c37f-4772-93e8-e52265db3493 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:55.965251 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ee9c62-f42b-4e81-b6ce-ab6272d86e7a admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:55.982228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e4001-285c-4227-a5bf-c17798481db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:55.983136 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e4001-285c-4227-a5bf-c17798481db4 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:55.983533 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e4001-285c-4227-a5bf-c17798481db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:55.984017 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e4001-285c-4227-a5bf-c17798481db4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:55.984756 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4e4001-285c-4227-a5bf-c17798481db4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:55.987198 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4e4001-285c-4227-a5bf-c17798481db4 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:56.022927 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:56.031709 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:56.045761 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 948/1878] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:42:55 2022] POST /identity/v3/projects => generated 379 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:56.046985 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 931/1879] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:42:55 2022] POST /identity/v3/projects => generated 365 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:56.051344 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34708deb-c37f-4772-93e8-e52265db3493 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:56.051504 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e4001-285c-4227-a5bf-c17798481db4 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:56.152886 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e4001-285c-4227-a5bf-c17798481db4 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:56.164034 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386669f1-fcc9-43dd-b0d4-919e82fdb61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:56.164718 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386669f1-fcc9-43dd-b0d4-919e82fdb61f admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:56.165057 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386669f1-fcc9-43dd-b0d4-919e82fdb61f admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:56.165340 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386669f1-fcc9-43dd-b0d4-919e82fdb61f admin admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:56.166472 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386669f1-fcc9-43dd-b0d4-919e82fdb61f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:56.171706 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386669f1-fcc9-43dd-b0d4-919e82fdb61f admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:56.189640 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34708deb-c37f-4772-93e8-e52265db3493 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:56.196905 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8698f70b-2191-479f-8a38-813cbdbbf844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:56.198304 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698f70b-2191-479f-8a38-813cbdbbf844 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:56.198794 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698f70b-2191-479f-8a38-813cbdbbf844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:56.199507 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698f70b-2191-479f-8a38-813cbdbbf844 admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:56.200421 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8698f70b-2191-479f-8a38-813cbdbbf844 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:56.204157 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8698f70b-2191-479f-8a38-813cbdbbf844 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:56.208276 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 932/1880] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:42:56 2022] POST /identity/v3/users => generated 283 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:56.211732 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386669f1-fcc9-43dd-b0d4-919e82fdb61f admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:56.225861 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386669f1-fcc9-43dd-b0d4-919e82fdb61f admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:56.234804 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad77f06b-43c5-45bd-944c-91c00a4ad2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:56.235428 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad77f06b-43c5-45bd-944c-91c00a4ad2cf admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:56.235878 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad77f06b-43c5-45bd-944c-91c00a4ad2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:56.236165 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad77f06b-43c5-45bd-944c-91c00a4ad2cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:56.236992 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad77f06b-43c5-45bd-944c-91c00a4ad2cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:56.238979 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad77f06b-43c5-45bd-944c-91c00a4ad2cf admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:56.244494 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 949/1881] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:42:56 2022] POST /identity/v3/users => generated 290 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:56.248196 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8698f70b-2191-479f-8a38-813cbdbbf844 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:56.251179 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 933/1882] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:56 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:56.254977 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad77f06b-43c5-45bd-944c-91c00a4ad2cf admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:56.262901 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8698f70b-2191-479f-8a38-813cbdbbf844 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:56.267059 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad77f06b-43c5-45bd-944c-91c00a4ad2cf admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:56.268382 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fea662d-8452-43fb-9b05-bef5409ff891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:56.269000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea662d-8452-43fb-9b05-bef5409ff891 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:56.269428 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea662d-8452-43fb-9b05-bef5409ff891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:56.269784 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea662d-8452-43fb-9b05-bef5409ff891 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:56.270584 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fea662d-8452-43fb-9b05-bef5409ff891 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:56.272994 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fea662d-8452-43fb-9b05-bef5409ff891 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:56.273560 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e296125-ddc1-4dda-89b5-bb96189923cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:56.274596 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e296125-ddc1-4dda-89b5-bb96189923cf admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:56.275199 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e296125-ddc1-4dda-89b5-bb96189923cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:56.275581 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e296125-ddc1-4dda-89b5-bb96189923cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:56.276388 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e296125-ddc1-4dda-89b5-bb96189923cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:56.278652 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e296125-ddc1-4dda-89b5-bb96189923cf admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:56.289435 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 950/1883] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:56 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:56.293088 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fea662d-8452-43fb-9b05-bef5409ff891 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:56.297402 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 934/1884] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:56 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:42:56.300943 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e296125-ddc1-4dda-89b5-bb96189923cf admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:56.312051 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fea662d-8452-43fb-9b05-bef5409ff891 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:56.313163 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e296125-ddc1-4dda-89b5-bb96189923cf admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:56.317990 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f0a396-ceb4-4dec-98f1-cecb870f87d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:56.318845 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb26967-85a2-4887-85df-68f3aeee3fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:56.319189 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0a396-ceb4-4dec-98f1-cecb870f87d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:56.319723 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb26967-85a2-4887-85df-68f3aeee3fc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:56.319841 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0a396-ceb4-4dec-98f1-cecb870f87d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:56.320264 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb26967-85a2-4887-85df-68f3aeee3fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:56.320549 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb26967-85a2-4887-85df-68f3aeee3fc8 admin admin] PATH_INFO: `/v3/projects/18d3284e74ab4dacae9bb1f907298d05/users/5c6aeb325a0c43aa98f435892699bbac/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:56.320801 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0a396-ceb4-4dec-98f1-cecb870f87d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:56.321866 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f0a396-ceb4-4dec-98f1-cecb870f87d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:56.324024 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f0a396-ceb4-4dec-98f1-cecb870f87d3 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:56.339994 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb26967-85a2-4887-85df-68f3aeee3fc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18d3284e74ab4dacae9bb1f907298d05, user_id=5c6aeb325a0c43aa98f435892699bbac, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:56.340235 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 951/1885] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:56 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:56.343566 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb26967-85a2-4887-85df-68f3aeee3fc8 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:56.344046 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f0a396-ceb4-4dec-98f1-cecb870f87d3 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:56.357506 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f0a396-ceb4-4dec-98f1-cecb870f87d3 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:56.371406 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 935/1886] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:42:56 2022] PUT /identity/v3/projects/18d3284e74ab4dacae9bb1f907298d05/users/5c6aeb325a0c43aa98f435892699bbac/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:56.374340 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e171fdc2-633e-4568-92a9-d5db3122bb0e None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:56.374871 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e171fdc2-633e-4568-92a9-d5db3122bb0e None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:56.375283 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e171fdc2-633e-4568-92a9-d5db3122bb0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:56.416164 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-e171fdc2-633e-4568-92a9-d5db3122bb0e None None] MFA Rules not processed for user `5c6aeb325a0c43aa98f435892699bbac`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:56.417511 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-e171fdc2-633e-4568-92a9-d5db3122bb0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:56.447225 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf6b6b6-214f-4574-9909-2452331640d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:56.448618 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf6b6b6-214f-4574-9909-2452331640d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:56.449406 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf6b6b6-214f-4574-9909-2452331640d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:56.449885 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf6b6b6-214f-4574-9909-2452331640d3 admin admin] PATH_INFO: `/v3/projects/86b78ba5cb474866a902bad41662a5ba/users/615bf5550f28423695ff5542a246a2fb/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:56.473712 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf6b6b6-214f-4574-9909-2452331640d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86b78ba5cb474866a902bad41662a5ba, user_id=615bf5550f28423695ff5542a246a2fb, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:56.478604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf6b6b6-214f-4574-9909-2452331640d3 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:56.512799 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 952/1887] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:42:56 2022] PUT /identity/v3/projects/86b78ba5cb474866a902bad41662a5ba/users/615bf5550f28423695ff5542a246a2fb/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:42:56.520252 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 936/1888] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:42:56.520475 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf6b6b6-214f-4574-9909-2452331640d3 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:56.522196 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6339d-9eb6-4ee1-8a85-5623a19d105f None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:56.522482 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6339d-9eb6-4ee1-8a85-5623a19d105f None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:56.522757 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6339d-9eb6-4ee1-8a85-5623a19d105f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:56.584742 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-d4c6339d-9eb6-4ee1-8a85-5623a19d105f None None] MFA Rules not processed for user `615bf5550f28423695ff5542a246a2fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:56.585988 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-d4c6339d-9eb6-4ee1-8a85-5623a19d105f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:56.605827 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:56.616398 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:56.636624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf6b6b6-214f-4574-9909-2452331640d3 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:56.642723 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3194e-3251-441e-bccd-dc1b2192517d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:56.644591 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3194e-3251-441e-bccd-dc1b2192517d admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:56.646142 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3194e-3251-441e-bccd-dc1b2192517d admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:56.646724 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3194e-3251-441e-bccd-dc1b2192517d admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:56.647619 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa3194e-3251-441e-bccd-dc1b2192517d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:56.649830 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa3194e-3251-441e-bccd-dc1b2192517d admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:56.697719 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 937/1889] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:42:56.698224 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 953/1890] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:42:56 2022] POST /identity/v3/projects => generated 365 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:56.698714 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c6339d-9eb6-4ee1-8a85-5623a19d105f None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:56.704027 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3194e-3251-441e-bccd-dc1b2192517d admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:56.721572 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c6339d-9eb6-4ee1-8a85-5623a19d105f None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:56.796897 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3194e-3251-441e-bccd-dc1b2192517d admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:56.802256 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ef54f5-4cb1-42e0-8ef1-00ac5794a60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:56.803514 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef54f5-4cb1-42e0-8ef1-00ac5794a60d admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:56.803867 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef54f5-4cb1-42e0-8ef1-00ac5794a60d admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:56.804197 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef54f5-4cb1-42e0-8ef1-00ac5794a60d admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:56.804918 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ef54f5-4cb1-42e0-8ef1-00ac5794a60d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:56.806836 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ef54f5-4cb1-42e0-8ef1-00ac5794a60d admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:56.847349 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ad0a4-81fe-4b5c-8709-2fc1b5239c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:56.848691 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ad0a4-81fe-4b5c-8709-2fc1b5239c56 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:56.848988 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ad0a4-81fe-4b5c-8709-2fc1b5239c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:56.849264 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ad0a4-81fe-4b5c-8709-2fc1b5239c56 admin admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:56.851979 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006ad0a4-81fe-4b5c-8709-2fc1b5239c56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:56.861220 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 954/1891] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:42:56 2022] POST /identity/v3/projects => generated 379 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:42:56.862464 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ef54f5-4cb1-42e0-8ef1-00ac5794a60d admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:56.864584 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006ad0a4-81fe-4b5c-8709-2fc1b5239c56 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:56.924117 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 938/1892] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:42:56 2022] POST /identity/v3/users => generated 289 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:56.927248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ad0a4-81fe-4b5c-8709-2fc1b5239c56 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:56.953087 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ad0a4-81fe-4b5c-8709-2fc1b5239c56 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:57.010104 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ef54f5-4cb1-42e0-8ef1-00ac5794a60d admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:57.030992 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae99f1-27f8-4a50-b8ae-3dcf98c5c201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:57.032665 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae99f1-27f8-4a50-b8ae-3dcf98c5c201 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:57.033304 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae99f1-27f8-4a50-b8ae-3dcf98c5c201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:57.033808 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae99f1-27f8-4a50-b8ae-3dcf98c5c201 admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:57.034938 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ae99f1-27f8-4a50-b8ae-3dcf98c5c201 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:57.038106 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ae99f1-27f8-4a50-b8ae-3dcf98c5c201 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:57.065749 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f4ec49-9ba2-4474-93da-9853ed6ff1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:57.066389 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4ec49-9ba2-4474-93da-9853ed6ff1fb admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:57.066685 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4ec49-9ba2-4474-93da-9853ed6ff1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:57.066950 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4ec49-9ba2-4474-93da-9853ed6ff1fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:57.067682 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f4ec49-9ba2-4474-93da-9853ed6ff1fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:57.069286 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f4ec49-9ba2-4474-93da-9853ed6ff1fb admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:57.077223 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 955/1893] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:42:56 2022] POST /identity/v3/users => generated 290 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:57.080529 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae99f1-27f8-4a50-b8ae-3dcf98c5c201 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:57.103513 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae99f1-27f8-4a50-b8ae-3dcf98c5c201 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:57.124778 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 939/1894] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:56 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:57.127498 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f4ec49-9ba2-4474-93da-9853ed6ff1fb admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:57.131042 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6536e9a7-b085-4e4e-a46e-f27df2959e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:57.133545 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536e9a7-b085-4e4e-a46e-f27df2959e5f admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:57.133741 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536e9a7-b085-4e4e-a46e-f27df2959e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:57.134001 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536e9a7-b085-4e4e-a46e-f27df2959e5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:57.134664 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6536e9a7-b085-4e4e-a46e-f27df2959e5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:57.137202 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6536e9a7-b085-4e4e-a46e-f27df2959e5f admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:57.154261 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 956/1895] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:57 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:42:57.158393 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6536e9a7-b085-4e4e-a46e-f27df2959e5f admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:57.158530 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f4ec49-9ba2-4474-93da-9853ed6ff1fb admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:57.173512 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7665174d-6281-452f-b1d1-ac3698761edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:57.174347 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6536e9a7-b085-4e4e-a46e-f27df2959e5f admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:57.174732 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7665174d-6281-452f-b1d1-ac3698761edb admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:57.175195 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7665174d-6281-452f-b1d1-ac3698761edb admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:57.175697 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7665174d-6281-452f-b1d1-ac3698761edb admin admin] PATH_INFO: `/v3/projects/f70f78fa5bcf4b75a2303091694e32c6/users/415d671b6c6543b497ae64154e139e56/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:57.182169 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b750ecc4-26ed-4d96-9688-c2177d82c283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:57.182635 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b750ecc4-26ed-4d96-9688-c2177d82c283 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:57.182925 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b750ecc4-26ed-4d96-9688-c2177d82c283 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:57.183202 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b750ecc4-26ed-4d96-9688-c2177d82c283 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:57.184667 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b750ecc4-26ed-4d96-9688-c2177d82c283 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:57.195308 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b750ecc4-26ed-4d96-9688-c2177d82c283 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:57.209120 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 957/1896] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:57 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:57.231559 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b750ecc4-26ed-4d96-9688-c2177d82c283 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:57.238157 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7665174d-6281-452f-b1d1-ac3698761edb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f70f78fa5bcf4b75a2303091694e32c6, user_id=415d671b6c6543b497ae64154e139e56, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:57.241458 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7665174d-6281-452f-b1d1-ac3698761edb admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:57.271665 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b750ecc4-26ed-4d96-9688-c2177d82c283 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:57.291101 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 940/1897] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:42:57 2022] PUT /identity/v3/projects/f70f78fa5bcf4b75a2303091694e32c6/users/415d671b6c6543b497ae64154e139e56/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:57.297054 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7665174d-6281-452f-b1d1-ac3698761edb admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:57.349759 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:57.365615 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:57.459456 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e444c-a59e-4fb1-b4ce-eb85dbf53457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:57.460145 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e444c-a59e-4fb1-b4ce-eb85dbf53457 admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:57.460422 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e444c-a59e-4fb1-b4ce-eb85dbf53457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:57.461529 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e444c-a59e-4fb1-b4ce-eb85dbf53457 admin admin] PATH_INFO: `/v3/projects/4f90d68eba3c4b698a2b2148027ca7e9/users/5532c544dc8e474ba9195571ae4414fb/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:57.541547 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456e444c-a59e-4fb1-b4ce-eb85dbf53457 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f90d68eba3c4b698a2b2148027ca7e9, user_id=5532c544dc8e474ba9195571ae4414fb, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:57.549824 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7665174d-6281-452f-b1d1-ac3698761edb admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:57.556659 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456e444c-a59e-4fb1-b4ce-eb85dbf53457 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:57.570386 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1593258-da5f-4a0f-baab-16fa2420d112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:57.572245 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1593258-da5f-4a0f-baab-16fa2420d112 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:57.575458 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1593258-da5f-4a0f-baab-16fa2420d112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:57.578288 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1593258-da5f-4a0f-baab-16fa2420d112 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:57.582257 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1593258-da5f-4a0f-baab-16fa2420d112 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:57.586730 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1593258-da5f-4a0f-baab-16fa2420d112 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:57.604481 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 958/1898] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:42:57 2022] PUT /identity/v3/projects/4f90d68eba3c4b698a2b2148027ca7e9/users/5532c544dc8e474ba9195571ae4414fb/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:57.608141 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e444c-a59e-4fb1-b4ce-eb85dbf53457 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:57.612236 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-456e444c-a59e-4fb1-b4ce-eb85dbf53457 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:57.619592 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 941/1899] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:57 2022] GET /identity/v3/roles => generated 1420 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:57.621075 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269df45-270c-48b6-9396-93ac34248f9c None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:57.621358 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269df45-270c-48b6-9396-93ac34248f9c None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:57.621632 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269df45-270c-48b6-9396-93ac34248f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:57.671189 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-2269df45-270c-48b6-9396-93ac34248f9c None None] MFA Rules not processed for user `5532c544dc8e474ba9195571ae4414fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:57.672991 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-2269df45-270c-48b6-9396-93ac34248f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:57.691829 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e444c-a59e-4fb1-b4ce-eb85dbf53457 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:57.695590 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26661068-26d8-4cc3-b680-f8f67a8c44b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:57.696195 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26661068-26d8-4cc3-b680-f8f67a8c44b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:57.696477 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26661068-26d8-4cc3-b680-f8f67a8c44b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:57.696747 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26661068-26d8-4cc3-b680-f8f67a8c44b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:57.698720 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-26661068-26d8-4cc3-b680-f8f67a8c44b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:57.784391 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 942/1900] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:57.785245 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2269df45-270c-48b6-9396-93ac34248f9c None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:57.791878 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26661068-26d8-4cc3-b680-f8f67a8c44b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:57.794245 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26661068-26d8-4cc3-b680-f8f67a8c44b2 service cinder] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:57.823056 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 959/1901] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:57.824703 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26661068-26d8-4cc3-b680-f8f67a8c44b2 service cinder] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:57.906984 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2269df45-270c-48b6-9396-93ac34248f9c None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:57.917357 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3bb956-f4da-4d69-a94c-c18c25e0fe81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:57.918226 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3bb956-f4da-4d69-a94c-c18c25e0fe81 admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:57.918632 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3bb956-f4da-4d69-a94c-c18c25e0fe81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:57.919000 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3bb956-f4da-4d69-a94c-c18c25e0fe81 admin admin] PATH_INFO: `/v3/projects/f70f78fa5bcf4b75a2303091694e32c6/users/415d671b6c6543b497ae64154e139e56/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:57.922492 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3bb956-f4da-4d69-a94c-c18c25e0fe81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f70f78fa5bcf4b75a2303091694e32c6, user_id=415d671b6c6543b497ae64154e139e56, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:57.926921 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3bb956-f4da-4d69-a94c-c18c25e0fe81 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:57.945860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 943/1902] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:42:57 2022] PUT /identity/v3/projects/f70f78fa5bcf4b75a2303091694e32c6/users/415d671b6c6543b497ae64154e139e56/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:57.957755 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032c235-4ad6-4c32-b5dc-0b717d3de155 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:57.958789 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032c235-4ad6-4c32-b5dc-0b717d3de155 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:57.959177 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032c235-4ad6-4c32-b5dc-0b717d3de155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:57.984948 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26661068-26d8-4cc3-b680-f8f67a8c44b2 service cinder] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:58.038724 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:58.046494 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-3032c235-4ad6-4c32-b5dc-0b717d3de155 None None] MFA Rules not processed for user `415d671b6c6543b497ae64154e139e56`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:58.048415 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-3032c235-4ad6-4c32-b5dc-0b717d3de155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:58.096119 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:58.171325 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 944/1903] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:57 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:42:58.183720 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3032c235-4ad6-4c32-b5dc-0b717d3de155 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:58.185376 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-3032c235-4ad6-4c32-b5dc-0b717d3de155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:58.191672 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbdafcf-695c-4a86-91db-1e80ef1f8f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:58.192763 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbdafcf-695c-4a86-91db-1e80ef1f8f8a admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:58.193051 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbdafcf-695c-4a86-91db-1e80ef1f8f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:58.193322 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbdafcf-695c-4a86-91db-1e80ef1f8f8a admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:58.193967 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbdafcf-695c-4a86-91db-1e80ef1f8f8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:58.195929 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbdafcf-695c-4a86-91db-1e80ef1f8f8a admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:58.277251 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 960/1904] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:42:57 2022] POST /identity/v3/projects => generated 379 bytes in 454 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:58.278209 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbdafcf-695c-4a86-91db-1e80ef1f8f8a admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:58.284718 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3032c235-4ad6-4c32-b5dc-0b717d3de155 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:58.376496 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf8f4a1-71a4-4426-96e6-c4d38f3b9d08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:58.378278 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8f4a1-71a4-4426-96e6-c4d38f3b9d08 service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:58.378864 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8f4a1-71a4-4426-96e6-c4d38f3b9d08 service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:58.379349 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8f4a1-71a4-4426-96e6-c4d38f3b9d08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:58.403789 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbdafcf-695c-4a86-91db-1e80ef1f8f8a admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:58.422641 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7fcefa-1d65-422b-b987-9bbc1c593220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:58.424448 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7fcefa-1d65-422b-b987-9bbc1c593220 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:58.424727 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7fcefa-1d65-422b-b987-9bbc1c593220 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:58.426468 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7fcefa-1d65-422b-b987-9bbc1c593220 admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:58.439952 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7fcefa-1d65-422b-b987-9bbc1c593220 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:58.439952 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7fcefa-1d65-422b-b987-9bbc1c593220 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:58.468319 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 961/1905] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:42:58 2022] POST /identity/v3/users => generated 296 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:42:58.472739 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7fcefa-1d65-422b-b987-9bbc1c593220 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:58.487576 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7fcefa-1d65-422b-b987-9bbc1c593220 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:58.502503 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cfd4e2-3575-483d-a41d-87e497610394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:58.503932 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cfd4e2-3575-483d-a41d-87e497610394 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:58.504169 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cfd4e2-3575-483d-a41d-87e497610394 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:58.504536 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cfd4e2-3575-483d-a41d-87e497610394 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:58.505274 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cfd4e2-3575-483d-a41d-87e497610394 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:58.507428 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf8f4a1-71a4-4426-96e6-c4d38f3b9d08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:58.507621 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cfd4e2-3575-483d-a41d-87e497610394 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:58.509252 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf8f4a1-71a4-4426-96e6-c4d38f3b9d08 service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:58.521699 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 945/1906] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:58 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:58.526679 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 962/1907] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:58 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:42:58.533893 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf8f4a1-71a4-4426-96e6-c4d38f3b9d08 service cinder] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:58.547168 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf8f4a1-71a4-4426-96e6-c4d38f3b9d08 service cinder] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:58.553265 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92308b7-4ab4-4a1a-b037-0432ef1e8b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:58.554752 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92308b7-4ab4-4a1a-b037-0432ef1e8b44 admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:58.555479 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92308b7-4ab4-4a1a-b037-0432ef1e8b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:58.555810 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92308b7-4ab4-4a1a-b037-0432ef1e8b44 admin admin] PATH_INFO: `/v3/projects/4099ad654a6049e8b15874ba261ac9fa/users/d4d214f107a74fcb8e20c854cf23ca4c/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:58.592581 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92308b7-4ab4-4a1a-b037-0432ef1e8b44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4099ad654a6049e8b15874ba261ac9fa, user_id=d4d214f107a74fcb8e20c854cf23ca4c, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:58.598028 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92308b7-4ab4-4a1a-b037-0432ef1e8b44 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:58.622814 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 946/1908] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:42:58 2022] PUT /identity/v3/projects/4099ad654a6049e8b15874ba261ac9fa/users/d4d214f107a74fcb8e20c854cf23ca4c/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:58.627434 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cfd4e2-3575-483d-a41d-87e497610394 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:58.645393 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cfd4e2-3575-483d-a41d-87e497610394 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:58.820996 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9156ed74-fcd5-4ed1-989f-9304d151cdac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:58.821751 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9156ed74-fcd5-4ed1-989f-9304d151cdac admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:58.822161 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9156ed74-fcd5-4ed1-989f-9304d151cdac admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:58.822503 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9156ed74-fcd5-4ed1-989f-9304d151cdac admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:58.823462 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9156ed74-fcd5-4ed1-989f-9304d151cdac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:58.826189 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9156ed74-fcd5-4ed1-989f-9304d151cdac admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:58.841845 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 963/1909] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:42:58 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:42:58.844715 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92308b7-4ab4-4a1a-b037-0432ef1e8b44 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:58.863733 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92308b7-4ab4-4a1a-b037-0432ef1e8b44 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:58.871557 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b21ee68-4edb-45e6-9631-28a4c1551815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:58.872225 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21ee68-4edb-45e6-9631-28a4c1551815 admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:58.872636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21ee68-4edb-45e6-9631-28a4c1551815 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:58.872920 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21ee68-4edb-45e6-9631-28a4c1551815 admin admin] PATH_INFO: `/v3/projects/4099ad654a6049e8b15874ba261ac9fa/users/d4d214f107a74fcb8e20c854cf23ca4c/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:58.885114 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b21ee68-4edb-45e6-9631-28a4c1551815 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4099ad654a6049e8b15874ba261ac9fa, user_id=d4d214f107a74fcb8e20c854cf23ca4c, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:58.885114 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b21ee68-4edb-45e6-9631-28a4c1551815 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:58.885114 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:58.922391 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 947/1910] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:42:58 2022] PUT /identity/v3/projects/4099ad654a6049e8b15874ba261ac9fa/users/d4d214f107a74fcb8e20c854cf23ca4c/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:42:58.934384 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00bd239-45f4-4fb1-87d8-641c80dedfa1 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:58.934812 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00bd239-45f4-4fb1-87d8-641c80dedfa1 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:58.937552 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00bd239-45f4-4fb1-87d8-641c80dedfa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:59.001217 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b21ee68-4edb-45e6-9631-28a4c1551815 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:59.008161 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:42:59.011981 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-f00bd239-45f4-4fb1-87d8-641c80dedfa1 None None] MFA Rules not processed for user `d4d214f107a74fcb8e20c854cf23ca4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:42:59.015682 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-f00bd239-45f4-4fb1-87d8-641c80dedfa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:59.110733 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b21ee68-4edb-45e6-9631-28a4c1551815 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:59.117703 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a44c1-44ca-4feb-b74e-339481f58474 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:59.118855 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a44c1-44ca-4feb-b74e-339481f58474 service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:59.119893 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a44c1-44ca-4feb-b74e-339481f58474 service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:59.120334 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a44c1-44ca-4feb-b74e-339481f58474 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:59.122374 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-439a44c1-44ca-4feb-b74e-339481f58474 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:42:59.150991 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 964/1911] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:42:58 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:42:59.166670 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00bd239-45f4-4fb1-87d8-641c80dedfa1 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:59.182121 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00bd239-45f4-4fb1-87d8-641c80dedfa1 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:59.187626 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd834c8e-4ace-4957-9ff3-d79af249eea1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:59.188341 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd834c8e-4ace-4957-9ff3-d79af249eea1 service cinder] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:59.188695 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd834c8e-4ace-4957-9ff3-d79af249eea1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:59.189039 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd834c8e-4ace-4957-9ff3-d79af249eea1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:59.201002 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd834c8e-4ace-4957-9ff3-d79af249eea1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:59.203363 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd834c8e-4ace-4957-9ff3-d79af249eea1 service cinder] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:59.225560 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 965/1912] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:59 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:42:59.258732 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439a44c1-44ca-4feb-b74e-339481f58474 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:59.271638 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439a44c1-44ca-4feb-b74e-339481f58474 service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:59.307897 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 948/1913] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:59 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:42:59.344204 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd834c8e-4ace-4957-9ff3-d79af249eea1 service cinder] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:59.354375 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd834c8e-4ace-4957-9ff3-d79af249eea1 service cinder] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:59.359360 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdee354-2775-424b-a746-9dd148427177 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:59.359975 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdee354-2775-424b-a746-9dd148427177 service cinder] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:59.360252 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdee354-2775-424b-a746-9dd148427177 service cinder] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:59.360524 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdee354-2775-424b-a746-9dd148427177 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:42:59.458501 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdee354-2775-424b-a746-9dd148427177 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:42:59.462464 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdee354-2775-424b-a746-9dd148427177 service cinder] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:42:59.501507 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 966/1914] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:42:59 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:42:59.838781 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a44c1-44ca-4feb-b74e-339481f58474 service cinder] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:42:59.983613 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a44c1-44ca-4feb-b74e-339481f58474 service cinder] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:42:59.998027 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535594e1-269d-4dd6-af5d-1939a061b6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:42:59.998818 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535594e1-269d-4dd6-af5d-1939a061b6df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:42:59.999102 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535594e1-269d-4dd6-af5d-1939a061b6df admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:42:59.999405 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535594e1-269d-4dd6-af5d-1939a061b6df admin admin] PATH_INFO: `/v3/users/5c6aeb325a0c43aa98f435892699bbac` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:00.005226 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535594e1-269d-4dd6-af5d-1939a061b6df admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c6aeb325a0c43aa98f435892699bbac)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:00.007788 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535594e1-269d-4dd6-af5d-1939a061b6df admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:00.133013 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:INSERT=1,keystone:SELECT=92,keystone:DELETE=7 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:00.136599 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-535594e1-269d-4dd6-af5d-1939a061b6df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c6aeb325a0c43aa98f435892699bbac'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:00.137720 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-535594e1-269d-4dd6-af5d-1939a061b6df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c6aeb325a0c43aa98f435892699bbac'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:00.153387 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-535594e1-269d-4dd6-af5d-1939a061b6df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c6aeb325a0c43aa98f435892699bbac'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:00.161256 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-535594e1-269d-4dd6-af5d-1939a061b6df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c6aeb325a0c43aa98f435892699bbac'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:00.169023 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-535594e1-269d-4dd6-af5d-1939a061b6df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c6aeb325a0c43aa98f435892699bbac'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:00.175145 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 949/1915] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:42:59 2022] DELETE /identity/v3/users/5c6aeb325a0c43aa98f435892699bbac => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:00.179786 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade72494-871a-4d11-aca0-415ea23ea7a2 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:00.180313 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade72494-871a-4d11-aca0-415ea23ea7a2 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:00.180313 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade72494-871a-4d11-aca0-415ea23ea7a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:00.201067 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-ade72494-871a-4d11-aca0-415ea23ea7a2 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:00.202807 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-ade72494-871a-4d11-aca0-415ea23ea7a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:00.275283 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 967/1916] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:00.285133 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535594e1-269d-4dd6-af5d-1939a061b6df admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:00.286731 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-535594e1-269d-4dd6-af5d-1939a061b6df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:00.353593 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535594e1-269d-4dd6-af5d-1939a061b6df admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:00.357339 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e42411-246a-48b6-8b1f-e6a3125455ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:00.358527 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e42411-246a-48b6-8b1f-e6a3125455ff service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:00.358649 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e42411-246a-48b6-8b1f-e6a3125455ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:00.358903 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e42411-246a-48b6-8b1f-e6a3125455ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:00.369762 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e42411-246a-48b6-8b1f-e6a3125455ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:00.371234 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e42411-246a-48b6-8b1f-e6a3125455ff service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:00.381947 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 950/1917] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:43:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:43:01.654418 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade72494-871a-4d11-aca0-415ea23ea7a2 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:01.656331 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-ade72494-871a-4d11-aca0-415ea23ea7a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:01.677557 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade72494-871a-4d11-aca0-415ea23ea7a2 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:01.694810 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49683023-cc31-4613-b7dd-329dbbce5bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:01.695675 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49683023-cc31-4613-b7dd-329dbbce5bb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:01.696713 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49683023-cc31-4613-b7dd-329dbbce5bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:01.708349 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49683023-cc31-4613-b7dd-329dbbce5bb7 admin admin] PATH_INFO: `/v3/projects/18d3284e74ab4dacae9bb1f907298d05` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:01.718008 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49683023-cc31-4613-b7dd-329dbbce5bb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18d3284e74ab4dacae9bb1f907298d05)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:01.722062 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49683023-cc31-4613-b7dd-329dbbce5bb7 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:01.783336 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:01.850170 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-49683023-cc31-4613-b7dd-329dbbce5bb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18d3284e74ab4dacae9bb1f907298d05', 'request_id': 'req-49683023-cc31-4613-b7dd-329dbbce5bb7'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:01.858786 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-49683023-cc31-4613-b7dd-329dbbce5bb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18d3284e74ab4dacae9bb1f907298d05', 'request_id': 'req-49683023-cc31-4613-b7dd-329dbbce5bb7'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:01.866517 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-49683023-cc31-4613-b7dd-329dbbce5bb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18d3284e74ab4dacae9bb1f907298d05', 'request_id': 'req-49683023-cc31-4613-b7dd-329dbbce5bb7'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:01.883187 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-49683023-cc31-4613-b7dd-329dbbce5bb7 admin admin] The token cache is being invalidate because project 18d3284e74ab4dacae9bb1f907298d05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:43:01.883544 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-49683023-cc31-4613-b7dd-329dbbce5bb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:01.884217 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-49683023-cc31-4613-b7dd-329dbbce5bb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:01.887121 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 968/1918] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:43:01 2022] DELETE /identity/v3/projects/18d3284e74ab4dacae9bb1f907298d05 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:01.890867 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e42411-246a-48b6-8b1f-e6a3125455ff service neutron] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:01.902034 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e42411-246a-48b6-8b1f-e6a3125455ff service neutron] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:02.020409 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cbc4c9-0f7b-458e-a74d-7b8bc2c58761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:02.021186 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cbc4c9-0f7b-458e-a74d-7b8bc2c58761 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:02.021455 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cbc4c9-0f7b-458e-a74d-7b8bc2c58761 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:02.021744 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cbc4c9-0f7b-458e-a74d-7b8bc2c58761 admin admin] PATH_INFO: `/v3/users/415d671b6c6543b497ae64154e139e56` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:02.023148 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cbc4c9-0f7b-458e-a74d-7b8bc2c58761 admin admin] RBAC: Authorizing `identity:delete_user(user_id=415d671b6c6543b497ae64154e139e56)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:02.038572 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cbc4c9-0f7b-458e-a74d-7b8bc2c58761 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:02.158937 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-72cbc4c9-0f7b-458e-a74d-7b8bc2c58761 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '415d671b6c6543b497ae64154e139e56'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:02.159748 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-72cbc4c9-0f7b-458e-a74d-7b8bc2c58761 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '415d671b6c6543b497ae64154e139e56'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:02.177874 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-72cbc4c9-0f7b-458e-a74d-7b8bc2c58761 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '415d671b6c6543b497ae64154e139e56'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:02.191463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e8fb0-a39f-4cd3-bc40-da6ee90d0617 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:02.191463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e8fb0-a39f-4cd3-bc40-da6ee90d0617 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:02.191463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e8fb0-a39f-4cd3-bc40-da6ee90d0617 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:02.191463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 969/1919] 10.222.0.30 () {54 vars in 921 bytes} [Tue Aug 30 14:43:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:43:02.194138 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:02.209763 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-72cbc4c9-0f7b-458e-a74d-7b8bc2c58761 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '415d671b6c6543b497ae64154e139e56'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:02.227356 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-72cbc4c9-0f7b-458e-a74d-7b8bc2c58761 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '415d671b6c6543b497ae64154e139e56'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:02.241318 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d1554-b1ec-488c-846a-0b1d4623b072 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:02.241929 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d1554-b1ec-488c-846a-0b1d4623b072 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:02.242375 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d1554-b1ec-488c-846a-0b1d4623b072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:02.258864 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 951/1920] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:43:01 2022] DELETE /identity/v3/users/415d671b6c6543b497ae64154e139e56 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:02.277374 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-506d1554-b1ec-488c-846a-0b1d4623b072 None None] MFA Rules not processed for user `8878de66a03044e38e507028853a0cd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:02.280210 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-506d1554-b1ec-488c-846a-0b1d4623b072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:02.446362 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 970/1921] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Aug 30 14:43:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:02.488371 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cbc4c9-0f7b-458e-a74d-7b8bc2c58761 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:02.490026 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-72cbc4c9-0f7b-458e-a74d-7b8bc2c58761 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:02.524030 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cbc4c9-0f7b-458e-a74d-7b8bc2c58761 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:02.532034 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d54a37-ac11-4f52-bd58-fdd0ad8a487a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:02.532757 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d54a37-ac11-4f52-bd58-fdd0ad8a487a service nova] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:02.533046 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d54a37-ac11-4f52-bd58-fdd0ad8a487a service nova] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:02.533391 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d54a37-ac11-4f52-bd58-fdd0ad8a487a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:02.553010 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d54a37-ac11-4f52-bd58-fdd0ad8a487a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:02.557819 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d54a37-ac11-4f52-bd58-fdd0ad8a487a service nova] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:02.580885 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 952/1922] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:43:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:43:02.909697 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506d1554-b1ec-488c-846a-0b1d4623b072 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:02.912306 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-506d1554-b1ec-488c-846a-0b1d4623b072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:02.963018 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d54a37-ac11-4f52-bd58-fdd0ad8a487a service nova] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:02.964738 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-56d54a37-ac11-4f52-bd58-fdd0ad8a487a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:03.004796 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506d1554-b1ec-488c-846a-0b1d4623b072 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:03.009631 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274eb773-3c8c-4f16-b8b7-202664870551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:03.010204 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274eb773-3c8c-4f16-b8b7-202664870551 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:03.010486 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274eb773-3c8c-4f16-b8b7-202664870551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:03.010758 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274eb773-3c8c-4f16-b8b7-202664870551 admin admin] PATH_INFO: `/v3/projects/f70f78fa5bcf4b75a2303091694e32c6` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:03.011952 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274eb773-3c8c-4f16-b8b7-202664870551 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f70f78fa5bcf4b75a2303091694e32c6)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:03.014135 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274eb773-3c8c-4f16-b8b7-202664870551 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:03.042385 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d54a37-ac11-4f52-bd58-fdd0ad8a487a service nova] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:03.048287 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb8c94a-a851-4a89-947a-9c12f05089a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:03.049024 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb8c94a-a851-4a89-947a-9c12f05089a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:03.049274 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb8c94a-a851-4a89-947a-9c12f05089a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:03.049552 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb8c94a-a851-4a89-947a-9c12f05089a9 admin admin] PATH_INFO: `/v3/users/615bf5550f28423695ff5542a246a2fb` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:03.050834 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb8c94a-a851-4a89-947a-9c12f05089a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=615bf5550f28423695ff5542a246a2fb)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:03.052935 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb8c94a-a851-4a89-947a-9c12f05089a9 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:03.058458 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:03.066850 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-274eb773-3c8c-4f16-b8b7-202664870551 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f70f78fa5bcf4b75a2303091694e32c6', 'request_id': 'req-274eb773-3c8c-4f16-b8b7-202664870551'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:03.073182 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-274eb773-3c8c-4f16-b8b7-202664870551 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f70f78fa5bcf4b75a2303091694e32c6', 'request_id': 'req-274eb773-3c8c-4f16-b8b7-202664870551'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:03.079761 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-274eb773-3c8c-4f16-b8b7-202664870551 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f70f78fa5bcf4b75a2303091694e32c6', 'request_id': 'req-274eb773-3c8c-4f16-b8b7-202664870551'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:03.093197 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-274eb773-3c8c-4f16-b8b7-202664870551 admin admin] The token cache is being invalidate because project f70f78fa5bcf4b75a2303091694e32c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:43:03.093488 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-274eb773-3c8c-4f16-b8b7-202664870551 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:03.094129 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-274eb773-3c8c-4f16-b8b7-202664870551 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:03.095968 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 971/1923] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:43:02 2022] DELETE /identity/v3/projects/f70f78fa5bcf4b75a2303091694e32c6 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:03.100718 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe7ee3-e270-40f0-840c-495ce507bc85 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:03.101062 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe7ee3-e270-40f0-840c-495ce507bc85 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:03.101339 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe7ee3-e270-40f0-840c-495ce507bc85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:03.103927 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-ffb8c94a-a851-4a89-947a-9c12f05089a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '615bf5550f28423695ff5542a246a2fb'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:03.104567 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-ffb8c94a-a851-4a89-947a-9c12f05089a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '615bf5550f28423695ff5542a246a2fb'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:03.117932 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-ffb8c94a-a851-4a89-947a-9c12f05089a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '615bf5550f28423695ff5542a246a2fb'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:03.121081 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-0abe7ee3-e270-40f0-840c-495ce507bc85 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:03.122287 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-0abe7ee3-e270-40f0-840c-495ce507bc85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:03.125630 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-ffb8c94a-a851-4a89-947a-9c12f05089a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '615bf5550f28423695ff5542a246a2fb'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:03.133260 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-ffb8c94a-a851-4a89-947a-9c12f05089a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '615bf5550f28423695ff5542a246a2fb'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:03.139521 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 953/1924] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:43:02 2022] DELETE /identity/v3/users/615bf5550f28423695ff5542a246a2fb => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:03.144009 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933735a6-a3d7-4907-b21c-de998fca0b88 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:03.144295 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933735a6-a3d7-4907-b21c-de998fca0b88 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:03.144567 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933735a6-a3d7-4907-b21c-de998fca0b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:03.163469 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-933735a6-a3d7-4907-b21c-de998fca0b88 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:03.164682 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-933735a6-a3d7-4907-b21c-de998fca0b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:03.183885 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 972/1925] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:03.193989 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334a2517-ed4d-4101-8ba8-67cd062e5d94 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:03.194275 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334a2517-ed4d-4101-8ba8-67cd062e5d94 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:03.194565 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334a2517-ed4d-4101-8ba8-67cd062e5d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:03.214389 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-334a2517-ed4d-4101-8ba8-67cd062e5d94 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:03.215673 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-334a2517-ed4d-4101-8ba8-67cd062e5d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:03.216845 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:03.241737 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 954/1926] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:43:03.248193 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933735a6-a3d7-4907-b21c-de998fca0b88 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:03.250378 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-933735a6-a3d7-4907-b21c-de998fca0b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:03.297177 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 973/1927] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:03.301245 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334a2517-ed4d-4101-8ba8-67cd062e5d94 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:03.316088 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334a2517-ed4d-4101-8ba8-67cd062e5d94 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:03.328007 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2582dd4-29bd-4db2-9cac-dc89223947c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:03.329158 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2582dd4-29bd-4db2-9cac-dc89223947c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:03.329634 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933735a6-a3d7-4907-b21c-de998fca0b88 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:03.329904 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2582dd4-29bd-4db2-9cac-dc89223947c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:03.330227 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2582dd4-29bd-4db2-9cac-dc89223947c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:03.331934 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2582dd4-29bd-4db2-9cac-dc89223947c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:43:03.332390 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2582dd4-29bd-4db2-9cac-dc89223947c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:03.333591 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef3902d-5923-4f12-b62f-b8560532cac1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:03.334197 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3902d-5923-4f12-b62f-b8560532cac1 service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:03.334478 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3902d-5923-4f12-b62f-b8560532cac1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:03.334619 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2582dd4-29bd-4db2-9cac-dc89223947c8 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:03.334760 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3902d-5923-4f12-b62f-b8560532cac1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:03.346827 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef3902d-5923-4f12-b62f-b8560532cac1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:03.348297 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef3902d-5923-4f12-b62f-b8560532cac1 service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:03.359672 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 955/1928] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:43:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:43:03.363285 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 974/1929] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 14:43:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:43:03.366531 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef3902d-5923-4f12-b62f-b8560532cac1 service neutron] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:03.379058 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef3902d-5923-4f12-b62f-b8560532cac1 service neutron] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:03.384918 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de16671f-7bc2-4e49-8bf0-b02dab244485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:03.389984 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de16671f-7bc2-4e49-8bf0-b02dab244485 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:03.389984 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de16671f-7bc2-4e49-8bf0-b02dab244485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:03.389984 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de16671f-7bc2-4e49-8bf0-b02dab244485 admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:03.389984 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de16671f-7bc2-4e49-8bf0-b02dab244485 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:03.389984 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de16671f-7bc2-4e49-8bf0-b02dab244485 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:03.439266 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 956/1930] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:43:03 2022] POST /identity/v3/projects => generated 363 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:03.443463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2582dd4-29bd-4db2-9cac-dc89223947c8 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:03.457968 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2582dd4-29bd-4db2-9cac-dc89223947c8 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:03.545534 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:03.564300 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dece5a78-9593-4ed6-8564-e543f1248031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:03.564940 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dece5a78-9593-4ed6-8564-e543f1248031 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:03.565549 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dece5a78-9593-4ed6-8564-e543f1248031 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:03.565832 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dece5a78-9593-4ed6-8564-e543f1248031 admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:03.566748 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dece5a78-9593-4ed6-8564-e543f1248031 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:03.569346 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dece5a78-9593-4ed6-8564-e543f1248031 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:03.602982 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 975/1931] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:43:03 2022] POST /identity/v3/users => generated 282 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:43:03.609497 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de16671f-7bc2-4e49-8bf0-b02dab244485 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:03.627493 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de16671f-7bc2-4e49-8bf0-b02dab244485 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:03.632812 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fd8603-0749-44c5-a2f7-67ee9205640c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:03.633526 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd8603-0749-44c5-a2f7-67ee9205640c admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:03.633916 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd8603-0749-44c5-a2f7-67ee9205640c admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:03.634292 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd8603-0749-44c5-a2f7-67ee9205640c admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:03.635229 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fd8603-0749-44c5-a2f7-67ee9205640c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:03.637322 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fd8603-0749-44c5-a2f7-67ee9205640c admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:03.650389 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dece5a78-9593-4ed6-8564-e543f1248031 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:03.653012 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-dece5a78-9593-4ed6-8564-e543f1248031 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:03.655529 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 957/1932] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:03 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:03.660013 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fd8603-0749-44c5-a2f7-67ee9205640c admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:03.673214 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fd8603-0749-44c5-a2f7-67ee9205640c admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:03.673426 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dece5a78-9593-4ed6-8564-e543f1248031 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:03.678761 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e939338c-3440-4d4b-86ba-34b8b0f73900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:03.680574 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e939338c-3440-4d4b-86ba-34b8b0f73900 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:03.680876 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e939338c-3440-4d4b-86ba-34b8b0f73900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:03.681150 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e939338c-3440-4d4b-86ba-34b8b0f73900 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:03.681939 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e939338c-3440-4d4b-86ba-34b8b0f73900 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:03.682045 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66910ea-35de-4e4b-af75-39a9f8f09920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:03.683488 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66910ea-35de-4e4b-af75-39a9f8f09920 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:03.683594 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66910ea-35de-4e4b-af75-39a9f8f09920 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:03.683664 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e939338c-3440-4d4b-86ba-34b8b0f73900 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:03.683994 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66910ea-35de-4e4b-af75-39a9f8f09920 admin admin] PATH_INFO: `/v3/users/2105fe65015f4ae9bbd025d7725b68ea` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:03.686174 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66910ea-35de-4e4b-af75-39a9f8f09920 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2105fe65015f4ae9bbd025d7725b68ea)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:03.691168 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66910ea-35de-4e4b-af75-39a9f8f09920 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:03.754006 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 958/1933] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:03 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:43:03.755084 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e939338c-3440-4d4b-86ba-34b8b0f73900 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:03.808357 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e939338c-3440-4d4b-86ba-34b8b0f73900 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:03.827625 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831c38ad-c571-405c-b76a-10b5d7aaa2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:03.831949 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c38ad-c571-405c-b76a-10b5d7aaa2ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:03.832256 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c38ad-c571-405c-b76a-10b5d7aaa2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:03.832537 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c38ad-c571-405c-b76a-10b5d7aaa2ed admin admin] PATH_INFO: `/v3/projects/98d83b7b8f624a83ba12ceb9c2fbec0a/users/d86f105235f14b3f9129f732fed4f85d/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:03.841331 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-c66910ea-35de-4e4b-af75-39a9f8f09920 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2105fe65015f4ae9bbd025d7725b68ea'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:03.843631 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-c66910ea-35de-4e4b-af75-39a9f8f09920 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2105fe65015f4ae9bbd025d7725b68ea'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:03.856937 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831c38ad-c571-405c-b76a-10b5d7aaa2ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=98d83b7b8f624a83ba12ceb9c2fbec0a, user_id=d86f105235f14b3f9129f732fed4f85d, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:03.862815 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-c66910ea-35de-4e4b-af75-39a9f8f09920 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2105fe65015f4ae9bbd025d7725b68ea'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:03.871185 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831c38ad-c571-405c-b76a-10b5d7aaa2ed admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:03.875878 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-c66910ea-35de-4e4b-af75-39a9f8f09920 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2105fe65015f4ae9bbd025d7725b68ea'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:03.877224 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-c66910ea-35de-4e4b-af75-39a9f8f09920 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2105fe65015f4ae9bbd025d7725b68ea'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:03.900599 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 959/1934] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:03 2022] PUT /identity/v3/projects/98d83b7b8f624a83ba12ceb9c2fbec0a/users/d86f105235f14b3f9129f732fed4f85d/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:03.904509 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831c38ad-c571-405c-b76a-10b5d7aaa2ed admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:03.908868 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-831c38ad-c571-405c-b76a-10b5d7aaa2ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:03.922797 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 976/1935] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:43:03 2022] DELETE /identity/v3/users/2105fe65015f4ae9bbd025d7725b68ea => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:43:03.924261 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fefc0-2167-4b57-a5eb-db914b052219 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:03.924866 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fefc0-2167-4b57-a5eb-db914b052219 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:03.925294 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fefc0-2167-4b57-a5eb-db914b052219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:03.991311 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-558fefc0-2167-4b57-a5eb-db914b052219 None None] MFA Rules not processed for user `d86f105235f14b3f9129f732fed4f85d`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:03.992713 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-558fefc0-2167-4b57-a5eb-db914b052219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:04.044349 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831c38ad-c571-405c-b76a-10b5d7aaa2ed admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:04.095110 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 977/1936] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:04.096441 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321310ae-1b41-40d6-a110-30d6161d9874 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:04.096727 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321310ae-1b41-40d6-a110-30d6161d9874 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:04.097003 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321310ae-1b41-40d6-a110-30d6161d9874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:04.097282 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:04.129084 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-321310ae-1b41-40d6-a110-30d6161d9874 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:04.130949 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-321310ae-1b41-40d6-a110-30d6161d9874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:04.138951 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdfa517-4f8b-439d-b77a-596d436bb71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:04.139829 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdfa517-4f8b-439d-b77a-596d436bb71c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:04.140113 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdfa517-4f8b-439d-b77a-596d436bb71c admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:04.140388 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdfa517-4f8b-439d-b77a-596d436bb71c admin admin] PATH_INFO: `/v3/users/eec36a9fb3194b2092603625dce5ab88` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:04.141634 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdfa517-4f8b-439d-b77a-596d436bb71c admin admin] RBAC: Authorizing `identity:delete_user(user_id=eec36a9fb3194b2092603625dce5ab88)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:04.143581 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdfa517-4f8b-439d-b77a-596d436bb71c admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:04.236666 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-5bdfa517-4f8b-439d-b77a-596d436bb71c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eec36a9fb3194b2092603625dce5ab88'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:04.237608 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-5bdfa517-4f8b-439d-b77a-596d436bb71c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eec36a9fb3194b2092603625dce5ab88'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:04.253942 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:04.254603 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-5bdfa517-4f8b-439d-b77a-596d436bb71c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eec36a9fb3194b2092603625dce5ab88'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:04.263811 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-5bdfa517-4f8b-439d-b77a-596d436bb71c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eec36a9fb3194b2092603625dce5ab88'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:04.272549 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-5bdfa517-4f8b-439d-b77a-596d436bb71c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eec36a9fb3194b2092603625dce5ab88'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:04.278370 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 978/1937] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:04.280629 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321310ae-1b41-40d6-a110-30d6161d9874 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:04.280810 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 960/1938] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:43:03 2022] DELETE /identity/v3/users/eec36a9fb3194b2092603625dce5ab88 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:04.281714 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdfa517-4f8b-439d-b77a-596d436bb71c admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:04.282837 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-321310ae-1b41-40d6-a110-30d6161d9874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:04.283054 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-5bdfa517-4f8b-439d-b77a-596d436bb71c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:04.300006 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321310ae-1b41-40d6-a110-30d6161d9874 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:04.359141 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdfa517-4f8b-439d-b77a-596d436bb71c admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:04.365340 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097c5281-d487-4bb4-8b37-21d0c9ddfb97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:04.365982 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c5281-d487-4bb4-8b37-21d0c9ddfb97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:04.366259 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c5281-d487-4bb4-8b37-21d0c9ddfb97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:04.367759 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c5281-d487-4bb4-8b37-21d0c9ddfb97 admin admin] PATH_INFO: `/v3/projects/86b78ba5cb474866a902bad41662a5ba` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:04.368975 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097c5281-d487-4bb4-8b37-21d0c9ddfb97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86b78ba5cb474866a902bad41662a5ba)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:04.371013 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097c5281-d487-4bb4-8b37-21d0c9ddfb97 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:04.377214 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e84ddc-38bc-4f53-9bb6-d8cdaa462bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:04.378353 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e84ddc-38bc-4f53-9bb6-d8cdaa462bf9 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:04.379367 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e84ddc-38bc-4f53-9bb6-d8cdaa462bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:04.379800 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e84ddc-38bc-4f53-9bb6-d8cdaa462bf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:04.380928 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e84ddc-38bc-4f53-9bb6-d8cdaa462bf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:04.383384 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e84ddc-38bc-4f53-9bb6-d8cdaa462bf9 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:04.430393 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 979/1939] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:43:04 2022] POST /identity/v3/projects => generated 365 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:04.432776 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf4119-3f22-4903-b29b-b6dad8a89ef5 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:04.432776 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf4119-3f22-4903-b29b-b6dad8a89ef5 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:04.433014 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf4119-3f22-4903-b29b-b6dad8a89ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:04.434018 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-097c5281-d487-4bb4-8b37-21d0c9ddfb97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86b78ba5cb474866a902bad41662a5ba', 'request_id': 'req-097c5281-d487-4bb4-8b37-21d0c9ddfb97'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:04.440177 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:04.444829 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-097c5281-d487-4bb4-8b37-21d0c9ddfb97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86b78ba5cb474866a902bad41662a5ba', 'request_id': 'req-097c5281-d487-4bb4-8b37-21d0c9ddfb97'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:04.457469 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-097c5281-d487-4bb4-8b37-21d0c9ddfb97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86b78ba5cb474866a902bad41662a5ba', 'request_id': 'req-097c5281-d487-4bb4-8b37-21d0c9ddfb97'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:04.468283 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-2bbf4119-3f22-4903-b29b-b6dad8a89ef5 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:04.469685 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-2bbf4119-3f22-4903-b29b-b6dad8a89ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:04.476800 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-097c5281-d487-4bb4-8b37-21d0c9ddfb97 admin admin] The token cache is being invalidate because project 86b78ba5cb474866a902bad41662a5ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:43:04.477206 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-097c5281-d487-4bb4-8b37-21d0c9ddfb97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:04.477756 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-097c5281-d487-4bb4-8b37-21d0c9ddfb97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:04.479374 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 961/1940] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:43:04 2022] DELETE /identity/v3/projects/86b78ba5cb474866a902bad41662a5ba => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:04.480133 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097c5281-d487-4bb4-8b37-21d0c9ddfb97 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:04.481384 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-097c5281-d487-4bb4-8b37-21d0c9ddfb97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:04.535081 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 980/1941] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:04.535859 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf4119-3f22-4903-b29b-b6dad8a89ef5 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:04.537153 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-2bbf4119-3f22-4903-b29b-b6dad8a89ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:04.547670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097c5281-d487-4bb4-8b37-21d0c9ddfb97 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:04.550110 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf4119-3f22-4903-b29b-b6dad8a89ef5 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:04.554949 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7324518-9f7e-4653-8b27-859fd6ccb209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:04.555590 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7324518-9f7e-4653-8b27-859fd6ccb209 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:04.555871 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7324518-9f7e-4653-8b27-859fd6ccb209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:04.556146 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7324518-9f7e-4653-8b27-859fd6ccb209 admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:04.556776 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7324518-9f7e-4653-8b27-859fd6ccb209 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:04.558711 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7324518-9f7e-4653-8b27-859fd6ccb209 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:04.582426 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 981/1942] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:43:04 2022] POST /identity/v3/users => generated 289 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:04.583643 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7324518-9f7e-4653-8b27-859fd6ccb209 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:04.584807 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-f7324518-9f7e-4653-8b27-859fd6ccb209 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:04.597232 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7324518-9f7e-4653-8b27-859fd6ccb209 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:04.600205 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab4b5eb-076b-4b70-98a1-5166bbed1f33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:04.600810 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4b5eb-076b-4b70-98a1-5166bbed1f33 service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:04.601084 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4b5eb-076b-4b70-98a1-5166bbed1f33 service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:04.601359 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4b5eb-076b-4b70-98a1-5166bbed1f33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:04.602007 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd3cd0-6d62-4c82-be4c-900bf17e2e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:04.602667 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd3cd0-6d62-4c82-be4c-900bf17e2e13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:04.602949 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd3cd0-6d62-4c82-be4c-900bf17e2e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:04.603060 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-4ab4b5eb-076b-4b70-98a1-5166bbed1f33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:04.603309 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd3cd0-6d62-4c82-be4c-900bf17e2e13 admin admin] PATH_INFO: `/v3/users/5532c544dc8e474ba9195571ae4414fb` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:04.604473 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcd3cd0-6d62-4c82-be4c-900bf17e2e13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5532c544dc8e474ba9195571ae4414fb)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:04.606549 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcd3cd0-6d62-4c82-be4c-900bf17e2e13 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:04.615125 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab4b5eb-076b-4b70-98a1-5166bbed1f33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:04.616562 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab4b5eb-076b-4b70-98a1-5166bbed1f33 service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:04.627071 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 962/1943] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:43:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:43:04.628078 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab4b5eb-076b-4b70-98a1-5166bbed1f33 service neutron] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:04.638105 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab4b5eb-076b-4b70-98a1-5166bbed1f33 service neutron] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:04.643809 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28858ee-006f-4d99-8997-c782a52d26c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:04.644479 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28858ee-006f-4d99-8997-c782a52d26c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:04.644761 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28858ee-006f-4d99-8997-c782a52d26c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:04.645029 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28858ee-006f-4d99-8997-c782a52d26c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:04.658064 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28858ee-006f-4d99-8997-c782a52d26c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:04.660378 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28858ee-006f-4d99-8997-c782a52d26c5 service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:04.662670 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-8bcd3cd0-6d62-4c82-be4c-900bf17e2e13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5532c544dc8e474ba9195571ae4414fb'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:04.663675 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-8bcd3cd0-6d62-4c82-be4c-900bf17e2e13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5532c544dc8e474ba9195571ae4414fb'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:04.668371 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:04.674478 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-8bcd3cd0-6d62-4c82-be4c-900bf17e2e13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5532c544dc8e474ba9195571ae4414fb'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:04.677581 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 963/1944] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:43:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:43:04.679353 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28858ee-006f-4d99-8997-c782a52d26c5 service neutron] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:04.691042 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28858ee-006f-4d99-8997-c782a52d26c5 service neutron] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:04.698416 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ee944-22a3-4bfa-9d24-9c5ee4a141f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:04.699649 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ee944-22a3-4bfa-9d24-9c5ee4a141f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:04.699830 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-8bcd3cd0-6d62-4c82-be4c-900bf17e2e13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5532c544dc8e474ba9195571ae4414fb'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:04.700357 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ee944-22a3-4bfa-9d24-9c5ee4a141f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:04.700576 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-8bcd3cd0-6d62-4c82-be4c-900bf17e2e13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5532c544dc8e474ba9195571ae4414fb'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:04.700677 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ee944-22a3-4bfa-9d24-9c5ee4a141f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:04.702739 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7ee944-22a3-4bfa-9d24-9c5ee4a141f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:04.706241 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7ee944-22a3-4bfa-9d24-9c5ee4a141f1 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:04.722318 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 964/1945] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:04 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:04.726301 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 982/1946] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:43:04 2022] DELETE /identity/v3/users/5532c544dc8e474ba9195571ae4414fb => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:04.726735 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ee944-22a3-4bfa-9d24-9c5ee4a141f1 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:04.728800 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-ef7ee944-22a3-4bfa-9d24-9c5ee4a141f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:04.743958 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ee944-22a3-4bfa-9d24-9c5ee4a141f1 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:04.841178 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea24bb0-3cc9-4845-8027-34a4d4f17f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:04.842440 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea24bb0-3cc9-4845-8027-34a4d4f17f2c admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:04.843059 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea24bb0-3cc9-4845-8027-34a4d4f17f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:04.843685 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea24bb0-3cc9-4845-8027-34a4d4f17f2c admin admin] PATH_INFO: `/v3/projects/bcbb99fcea244c11bbe902c50b3f1614/users/fb1f84441c2f46bfb50da9e58ddb0a7b/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:04.863877 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea24bb0-3cc9-4845-8027-34a4d4f17f2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcbb99fcea244c11bbe902c50b3f1614, user_id=fb1f84441c2f46bfb50da9e58ddb0a7b, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:04.867526 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea24bb0-3cc9-4845-8027-34a4d4f17f2c admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:04.893574 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:04.916318 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd3cd0-6d62-4c82-be4c-900bf17e2e13 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:04.916712 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 965/1947] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:04 2022] PUT /identity/v3/projects/bcbb99fcea244c11bbe902c50b3f1614/users/fb1f84441c2f46bfb50da9e58ddb0a7b/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:43:04.932751 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd3cd0-6d62-4c82-be4c-900bf17e2e13 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:05.055322 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcfb206-2d62-44f3-ac10-56f91cdcd703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:05.055958 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcfb206-2d62-44f3-ac10-56f91cdcd703 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:05.056239 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcfb206-2d62-44f3-ac10-56f91cdcd703 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:05.056519 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcfb206-2d62-44f3-ac10-56f91cdcd703 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:05.057169 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcfb206-2d62-44f3-ac10-56f91cdcd703 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:05.058815 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcfb206-2d62-44f3-ac10-56f91cdcd703 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:05.070843 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 983/1948] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:04 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:43:05.075332 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea24bb0-3cc9-4845-8027-34a4d4f17f2c admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:05.086687 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea24bb0-3cc9-4845-8027-34a4d4f17f2c admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:05.091906 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd54ad9e-034e-41e3-8631-0ea6b8ed41d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:05.092358 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd54ad9e-034e-41e3-8631-0ea6b8ed41d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:05.092635 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd54ad9e-034e-41e3-8631-0ea6b8ed41d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:05.092907 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd54ad9e-034e-41e3-8631-0ea6b8ed41d4 admin admin] PATH_INFO: `/v3/projects/bcbb99fcea244c11bbe902c50b3f1614/users/fb1f84441c2f46bfb50da9e58ddb0a7b/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:05.094867 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd54ad9e-034e-41e3-8631-0ea6b8ed41d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcbb99fcea244c11bbe902c50b3f1614, user_id=fb1f84441c2f46bfb50da9e58ddb0a7b, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:05.097556 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd54ad9e-034e-41e3-8631-0ea6b8ed41d4 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:05.109778 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 966/1949] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:05 2022] PUT /identity/v3/projects/bcbb99fcea244c11bbe902c50b3f1614/users/fb1f84441c2f46bfb50da9e58ddb0a7b/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:05.120128 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459f39b-2fbb-43fe-b5d8-342acc158ac9 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:05.120410 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459f39b-2fbb-43fe-b5d8-342acc158ac9 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:05.120687 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459f39b-2fbb-43fe-b5d8-342acc158ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:05.149908 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-6459f39b-2fbb-43fe-b5d8-342acc158ac9 None None] MFA Rules not processed for user `fb1f84441c2f46bfb50da9e58ddb0a7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:05.151176 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-6459f39b-2fbb-43fe-b5d8-342acc158ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:05.257502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 984/1950] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:05 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:05.265601 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd54ad9e-034e-41e3-8631-0ea6b8ed41d4 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:05.272429 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-fd54ad9e-034e-41e3-8631-0ea6b8ed41d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:05.353622 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd54ad9e-034e-41e3-8631-0ea6b8ed41d4 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:05.358688 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4a7bcb-64c1-4a84-8dcf-5ac1524ad925 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:05.359565 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a7bcb-64c1-4a84-8dcf-5ac1524ad925 service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:05.359692 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a7bcb-64c1-4a84-8dcf-5ac1524ad925 service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:05.360072 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a7bcb-64c1-4a84-8dcf-5ac1524ad925 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:05.372371 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4a7bcb-64c1-4a84-8dcf-5ac1524ad925 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:05.374806 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4a7bcb-64c1-4a84-8dcf-5ac1524ad925 service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:05.386585 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 967/1951] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:43:05 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:43:05.473425 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6459f39b-2fbb-43fe-b5d8-342acc158ac9 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:05.475822 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-6459f39b-2fbb-43fe-b5d8-342acc158ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:05.503102 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4a7bcb-64c1-4a84-8dcf-5ac1524ad925 service cinder] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:05.518164 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4a7bcb-64c1-4a84-8dcf-5ac1524ad925 service cinder] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:05.535466 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386cdbd3-c86c-4520-99ed-63d6173326a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:05.536883 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386cdbd3-c86c-4520-99ed-63d6173326a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:05.537265 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386cdbd3-c86c-4520-99ed-63d6173326a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:05.537828 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386cdbd3-c86c-4520-99ed-63d6173326a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:05.548535 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-386cdbd3-c86c-4520-99ed-63d6173326a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:05.657496 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:05.677531 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6459f39b-2fbb-43fe-b5d8-342acc158ac9 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:05.683184 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118dafee-3a71-44f3-be90-b16141e3ec14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:05.684061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118dafee-3a71-44f3-be90-b16141e3ec14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:05.684421 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118dafee-3a71-44f3-be90-b16141e3ec14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:05.684768 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118dafee-3a71-44f3-be90-b16141e3ec14 admin admin] PATH_INFO: `/v3/projects/c4880227055646de88afca1aa55a2f90` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:05.686558 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118dafee-3a71-44f3-be90-b16141e3ec14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4880227055646de88afca1aa55a2f90)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:05.689469 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118dafee-3a71-44f3-be90-b16141e3ec14 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:05.693226 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386cdbd3-c86c-4520-99ed-63d6173326a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:05.694689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386cdbd3-c86c-4520-99ed-63d6173326a7 service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:05.709062 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 968/1952] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:43:05 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:43:05.709958 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386cdbd3-c86c-4520-99ed-63d6173326a7 service cinder] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:05.711546 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-386cdbd3-c86c-4520-99ed-63d6173326a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:05.734778 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386cdbd3-c86c-4520-99ed-63d6173326a7 service cinder] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:05.741893 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b557cf-207c-4897-a989-099e23962429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:05.743133 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b557cf-207c-4897-a989-099e23962429 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:05.743470 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b557cf-207c-4897-a989-099e23962429 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:05.743741 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b557cf-207c-4897-a989-099e23962429 admin admin] PATH_INFO: `/v3/projects/05aae87b4d6d4b30b43b263141a33007` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:05.746259 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b557cf-207c-4897-a989-099e23962429 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05aae87b4d6d4b30b43b263141a33007)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:05.748923 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b557cf-207c-4897-a989-099e23962429 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:05.781041 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-118dafee-3a71-44f3-be90-b16141e3ec14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4880227055646de88afca1aa55a2f90', 'request_id': 'req-118dafee-3a71-44f3-be90-b16141e3ec14'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:05.793081 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-118dafee-3a71-44f3-be90-b16141e3ec14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4880227055646de88afca1aa55a2f90', 'request_id': 'req-118dafee-3a71-44f3-be90-b16141e3ec14'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:05.813972 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-118dafee-3a71-44f3-be90-b16141e3ec14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4880227055646de88afca1aa55a2f90', 'request_id': 'req-118dafee-3a71-44f3-be90-b16141e3ec14'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:05.855468 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b6b557cf-207c-4897-a989-099e23962429 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05aae87b4d6d4b30b43b263141a33007', 'request_id': 'req-b6b557cf-207c-4897-a989-099e23962429'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:05.860801 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-118dafee-3a71-44f3-be90-b16141e3ec14 admin admin] The token cache is being invalidate because project c4880227055646de88afca1aa55a2f90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:43:05.864506 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-118dafee-3a71-44f3-be90-b16141e3ec14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:05.866598 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-118dafee-3a71-44f3-be90-b16141e3ec14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:05.874876 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 985/1953] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:43:05 2022] DELETE /identity/v3/projects/c4880227055646de88afca1aa55a2f90 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:05.874876 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118dafee-3a71-44f3-be90-b16141e3ec14 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:05.876333 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b6b557cf-207c-4897-a989-099e23962429 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05aae87b4d6d4b30b43b263141a33007', 'request_id': 'req-b6b557cf-207c-4897-a989-099e23962429'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:05.879712 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-118dafee-3a71-44f3-be90-b16141e3ec14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:05.945470 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b6b557cf-207c-4897-a989-099e23962429 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05aae87b4d6d4b30b43b263141a33007', 'request_id': 'req-b6b557cf-207c-4897-a989-099e23962429'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:05.967491 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b6b557cf-207c-4897-a989-099e23962429 admin admin] The token cache is being invalidate because project 05aae87b4d6d4b30b43b263141a33007 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:43:05.967807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b6b557cf-207c-4897-a989-099e23962429 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:05.968545 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b6b557cf-207c-4897-a989-099e23962429 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:05.970274 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 969/1954] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:43:05 2022] DELETE /identity/v3/projects/05aae87b4d6d4b30b43b263141a33007 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:05.971084 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b557cf-207c-4897-a989-099e23962429 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:05.976752 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-b6b557cf-207c-4897-a989-099e23962429 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:05.995067 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118dafee-3a71-44f3-be90-b16141e3ec14 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:05.995691 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:06.078245 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a8efcb-cb77-4be0-a4b2-bfca1f7e8997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:06.079086 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a8efcb-cb77-4be0-a4b2-bfca1f7e8997 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:06.079550 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a8efcb-cb77-4be0-a4b2-bfca1f7e8997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:06.080215 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a8efcb-cb77-4be0-a4b2-bfca1f7e8997 admin admin] PATH_INFO: `/v3/projects/4f90d68eba3c4b698a2b2148027ca7e9` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:06.083629 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a8efcb-cb77-4be0-a4b2-bfca1f7e8997 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f90d68eba3c4b698a2b2148027ca7e9)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:06.085861 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a8efcb-cb77-4be0-a4b2-bfca1f7e8997 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:06.098059 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b557cf-207c-4897-a989-099e23962429 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:06.105330 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ed970-3972-44ca-89b9-d9d2ffdb37a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:06.106839 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ed970-3972-44ca-89b9-d9d2ffdb37a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:06.107348 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ed970-3972-44ca-89b9-d9d2ffdb37a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:06.108561 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ed970-3972-44ca-89b9-d9d2ffdb37a9 admin admin] PATH_INFO: `/v3/users/d9a194c87e554a858beab1b03709e146` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:06.110511 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9ed970-3972-44ca-89b9-d9d2ffdb37a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9a194c87e554a858beab1b03709e146)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:06.113396 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9ed970-3972-44ca-89b9-d9d2ffdb37a9 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:06.136387 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:06.162247 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-12a8efcb-cb77-4be0-a4b2-bfca1f7e8997 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f90d68eba3c4b698a2b2148027ca7e9', 'request_id': 'req-12a8efcb-cb77-4be0-a4b2-bfca1f7e8997'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:06.172937 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-12a8efcb-cb77-4be0-a4b2-bfca1f7e8997 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f90d68eba3c4b698a2b2148027ca7e9', 'request_id': 'req-12a8efcb-cb77-4be0-a4b2-bfca1f7e8997'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:06.182003 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-12a8efcb-cb77-4be0-a4b2-bfca1f7e8997 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f90d68eba3c4b698a2b2148027ca7e9', 'request_id': 'req-12a8efcb-cb77-4be0-a4b2-bfca1f7e8997'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:06.198364 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-ca9ed970-3972-44ca-89b9-d9d2ffdb37a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9a194c87e554a858beab1b03709e146'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:06.199470 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-ca9ed970-3972-44ca-89b9-d9d2ffdb37a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9a194c87e554a858beab1b03709e146'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:06.206892 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-12a8efcb-cb77-4be0-a4b2-bfca1f7e8997 admin admin] The token cache is being invalidate because project 4f90d68eba3c4b698a2b2148027ca7e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:43:06.207301 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-12a8efcb-cb77-4be0-a4b2-bfca1f7e8997 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:06.209383 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-12a8efcb-cb77-4be0-a4b2-bfca1f7e8997 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:06.215498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 986/1955] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:43:05 2022] DELETE /identity/v3/projects/4f90d68eba3c4b698a2b2148027ca7e9 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:06.220556 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a8efcb-cb77-4be0-a4b2-bfca1f7e8997 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:06.224870 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-12a8efcb-cb77-4be0-a4b2-bfca1f7e8997 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:06.230965 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-ca9ed970-3972-44ca-89b9-d9d2ffdb37a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9a194c87e554a858beab1b03709e146'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:06.241454 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-ca9ed970-3972-44ca-89b9-d9d2ffdb37a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9a194c87e554a858beab1b03709e146'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:06.259362 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-ca9ed970-3972-44ca-89b9-d9d2ffdb37a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9a194c87e554a858beab1b03709e146'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:06.273857 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 970/1956] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:43:05 2022] DELETE /identity/v3/users/d9a194c87e554a858beab1b03709e146 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:06.273857 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ed970-3972-44ca-89b9-d9d2ffdb37a9 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:06.275972 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-ca9ed970-3972-44ca-89b9-d9d2ffdb37a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:06.318334 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a8efcb-cb77-4be0-a4b2-bfca1f7e8997 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:06.375663 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ed970-3972-44ca-89b9-d9d2ffdb37a9 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:06.382336 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9590311a-42f7-4016-b278-4cd4c1efa7e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:06.383236 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9590311a-42f7-4016-b278-4cd4c1efa7e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:06.383414 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9590311a-42f7-4016-b278-4cd4c1efa7e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:06.383960 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9590311a-42f7-4016-b278-4cd4c1efa7e7 admin admin] PATH_INFO: `/v3/users/d4d214f107a74fcb8e20c854cf23ca4c` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:06.385980 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9590311a-42f7-4016-b278-4cd4c1efa7e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4d214f107a74fcb8e20c854cf23ca4c)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:06.388056 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9590311a-42f7-4016-b278-4cd4c1efa7e7 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:06.405410 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb3a5c-8757-4a95-9929-8032144eeb3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:06.406458 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb3a5c-8757-4a95-9929-8032144eeb3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:06.406746 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb3a5c-8757-4a95-9929-8032144eeb3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:06.407085 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb3a5c-8757-4a95-9929-8032144eeb3e admin admin] PATH_INFO: `/v3/users/4338ff86945549d8916747a3605eca5b` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:06.408379 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fb3a5c-8757-4a95-9929-8032144eeb3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4338ff86945549d8916747a3605eca5b)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:06.411387 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fb3a5c-8757-4a95-9929-8032144eeb3e admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:06.432890 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:06.458608 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-9590311a-42f7-4016-b278-4cd4c1efa7e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4d214f107a74fcb8e20c854cf23ca4c'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:06.459256 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-9590311a-42f7-4016-b278-4cd4c1efa7e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4d214f107a74fcb8e20c854cf23ca4c'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:06.477199 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-9590311a-42f7-4016-b278-4cd4c1efa7e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4d214f107a74fcb8e20c854cf23ca4c'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:06.481059 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-a7fb3a5c-8757-4a95-9929-8032144eeb3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4338ff86945549d8916747a3605eca5b'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:06.483179 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-a7fb3a5c-8757-4a95-9929-8032144eeb3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4338ff86945549d8916747a3605eca5b'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:06.485731 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-9590311a-42f7-4016-b278-4cd4c1efa7e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4d214f107a74fcb8e20c854cf23ca4c'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:06.492529 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-a7fb3a5c-8757-4a95-9929-8032144eeb3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4338ff86945549d8916747a3605eca5b'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:06.497680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-9590311a-42f7-4016-b278-4cd4c1efa7e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4d214f107a74fcb8e20c854cf23ca4c'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:06.506019 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-a7fb3a5c-8757-4a95-9929-8032144eeb3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4338ff86945549d8916747a3605eca5b'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:06.507863 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-a7fb3a5c-8757-4a95-9929-8032144eeb3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4338ff86945549d8916747a3605eca5b'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:06.511801 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 971/1957] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:43:06 2022] DELETE /identity/v3/users/d4d214f107a74fcb8e20c854cf23ca4c => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:06.516341 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:06.551802 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 987/1958] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:43:06 2022] DELETE /identity/v3/users/4338ff86945549d8916747a3605eca5b => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:06.758016 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9590311a-42f7-4016-b278-4cd4c1efa7e7 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:06.759759 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-9590311a-42f7-4016-b278-4cd4c1efa7e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:06.857071 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9590311a-42f7-4016-b278-4cd4c1efa7e7 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:06.930652 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aacec3-3f2b-4e0c-8208-068a52df3767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:06.931459 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aacec3-3f2b-4e0c-8208-068a52df3767 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:06.931842 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aacec3-3f2b-4e0c-8208-068a52df3767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:06.932568 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aacec3-3f2b-4e0c-8208-068a52df3767 admin admin] PATH_INFO: `/v3/projects/2c07589ecf8e4e3cb4094d129d3bb890` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:06.933561 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aacec3-3f2b-4e0c-8208-068a52df3767 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c07589ecf8e4e3cb4094d129d3bb890)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:06.936445 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aacec3-3f2b-4e0c-8208-068a52df3767 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:07.049948 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-17aacec3-3f2b-4e0c-8208-068a52df3767 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c07589ecf8e4e3cb4094d129d3bb890', 'request_id': 'req-17aacec3-3f2b-4e0c-8208-068a52df3767'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:07.057223 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-17aacec3-3f2b-4e0c-8208-068a52df3767 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c07589ecf8e4e3cb4094d129d3bb890', 'request_id': 'req-17aacec3-3f2b-4e0c-8208-068a52df3767'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:07.062270 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:07.087933 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-17aacec3-3f2b-4e0c-8208-068a52df3767 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c07589ecf8e4e3cb4094d129d3bb890', 'request_id': 'req-17aacec3-3f2b-4e0c-8208-068a52df3767'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:07.101107 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb3a5c-8757-4a95-9929-8032144eeb3e admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:07.103797 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-a7fb3a5c-8757-4a95-9929-8032144eeb3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:07.122314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-17aacec3-3f2b-4e0c-8208-068a52df3767 admin admin] The token cache is being invalidate because project 2c07589ecf8e4e3cb4094d129d3bb890 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:43:07.123248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-17aacec3-3f2b-4e0c-8208-068a52df3767 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:07.131649 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-17aacec3-3f2b-4e0c-8208-068a52df3767 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:07.132590 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb3a5c-8757-4a95-9929-8032144eeb3e admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:07.135995 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 972/1959] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:43:06 2022] DELETE /identity/v3/projects/2c07589ecf8e4e3cb4094d129d3bb890 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:07.137268 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aacec3-3f2b-4e0c-8208-068a52df3767 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:07.152332 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-17aacec3-3f2b-4e0c-8208-068a52df3767 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:07.211591 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a72b87-9170-404a-ac1f-c4c2297a8570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:07.212670 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a72b87-9170-404a-ac1f-c4c2297a8570 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:07.213135 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a72b87-9170-404a-ac1f-c4c2297a8570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:07.213403 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a72b87-9170-404a-ac1f-c4c2297a8570 admin admin] PATH_INFO: `/v3/projects/4099ad654a6049e8b15874ba261ac9fa` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:07.214558 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a72b87-9170-404a-ac1f-c4c2297a8570 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4099ad654a6049e8b15874ba261ac9fa)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:07.217341 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a72b87-9170-404a-ac1f-c4c2297a8570 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:07.271136 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aacec3-3f2b-4e0c-8208-068a52df3767 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:07.272588 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-24a72b87-9170-404a-ac1f-c4c2297a8570 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4099ad654a6049e8b15874ba261ac9fa', 'request_id': 'req-24a72b87-9170-404a-ac1f-c4c2297a8570'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:07.281414 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-24a72b87-9170-404a-ac1f-c4c2297a8570 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4099ad654a6049e8b15874ba261ac9fa', 'request_id': 'req-24a72b87-9170-404a-ac1f-c4c2297a8570'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:07.288056 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-24a72b87-9170-404a-ac1f-c4c2297a8570 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4099ad654a6049e8b15874ba261ac9fa', 'request_id': 'req-24a72b87-9170-404a-ac1f-c4c2297a8570'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:07.303544 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-24a72b87-9170-404a-ac1f-c4c2297a8570 admin admin] The token cache is being invalidate because project 4099ad654a6049e8b15874ba261ac9fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:43:07.303927 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-24a72b87-9170-404a-ac1f-c4c2297a8570 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:07.304591 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-24a72b87-9170-404a-ac1f-c4c2297a8570 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:07.306660 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 988/1960] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:43:07 2022] DELETE /identity/v3/projects/4099ad654a6049e8b15874ba261ac9fa => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:07.311060 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a72b87-9170-404a-ac1f-c4c2297a8570 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:07.311060 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-24a72b87-9170-404a-ac1f-c4c2297a8570 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:07.345960 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980b0d41-c084-46d9-808b-91c3a0f7eddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:07.346729 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b0d41-c084-46d9-808b-91c3a0f7eddf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:07.347017 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b0d41-c084-46d9-808b-91c3a0f7eddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:07.347480 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b0d41-c084-46d9-808b-91c3a0f7eddf admin admin] PATH_INFO: `/v3/projects/c81a7c0942ef49e8825bdf58bc33a254` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:07.356135 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980b0d41-c084-46d9-808b-91c3a0f7eddf admin admin] RBAC: Authorizing `identity:delete_project(project_id=c81a7c0942ef49e8825bdf58bc33a254)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:07.365788 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980b0d41-c084-46d9-808b-91c3a0f7eddf admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:07.407772 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a72b87-9170-404a-ac1f-c4c2297a8570 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:07.416401 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9220b919-979b-455e-912d-33f3e630437f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:07.417846 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220b919-979b-455e-912d-33f3e630437f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:07.418207 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220b919-979b-455e-912d-33f3e630437f admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:07.418561 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220b919-979b-455e-912d-33f3e630437f admin admin] PATH_INFO: `/v3/users/cbf3579558d54be788f4030e3bc5d053` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:07.420488 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9220b919-979b-455e-912d-33f3e630437f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbf3579558d54be788f4030e3bc5d053)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:07.422594 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9220b919-979b-455e-912d-33f3e630437f admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:07.463392 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:07.502106 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-980b0d41-c084-46d9-808b-91c3a0f7eddf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c81a7c0942ef49e8825bdf58bc33a254', 'request_id': 'req-980b0d41-c084-46d9-808b-91c3a0f7eddf'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:07.520987 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-980b0d41-c084-46d9-808b-91c3a0f7eddf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c81a7c0942ef49e8825bdf58bc33a254', 'request_id': 'req-980b0d41-c084-46d9-808b-91c3a0f7eddf'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:07.536565 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-980b0d41-c084-46d9-808b-91c3a0f7eddf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c81a7c0942ef49e8825bdf58bc33a254', 'request_id': 'req-980b0d41-c084-46d9-808b-91c3a0f7eddf'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:07.551102 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-9220b919-979b-455e-912d-33f3e630437f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbf3579558d54be788f4030e3bc5d053'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:07.552979 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-9220b919-979b-455e-912d-33f3e630437f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbf3579558d54be788f4030e3bc5d053'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:07.568386 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-9220b919-979b-455e-912d-33f3e630437f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbf3579558d54be788f4030e3bc5d053'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:07.578570 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-980b0d41-c084-46d9-808b-91c3a0f7eddf admin admin] The token cache is being invalidate because project c81a7c0942ef49e8825bdf58bc33a254 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:43:07.579087 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-980b0d41-c084-46d9-808b-91c3a0f7eddf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:07.581141 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-980b0d41-c084-46d9-808b-91c3a0f7eddf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:07.585468 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 973/1961] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:43:07 2022] DELETE /identity/v3/projects/c81a7c0942ef49e8825bdf58bc33a254 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:07.586680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011208b3-25a5-42e2-8dfa-885dc7f1c94c None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:07.588115 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011208b3-25a5-42e2-8dfa-885dc7f1c94c None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:07.588471 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011208b3-25a5-42e2-8dfa-885dc7f1c94c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:07.599921 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-9220b919-979b-455e-912d-33f3e630437f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbf3579558d54be788f4030e3bc5d053'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:07.600494 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-9220b919-979b-455e-912d-33f3e630437f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbf3579558d54be788f4030e3bc5d053'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:07.626454 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-011208b3-25a5-42e2-8dfa-885dc7f1c94c None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:07.628531 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-011208b3-25a5-42e2-8dfa-885dc7f1c94c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:07.634586 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 989/1962] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:43:07 2022] DELETE /identity/v3/users/cbf3579558d54be788f4030e3bc5d053 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:07.641152 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e5ffa-3a7d-4cc3-a7f2-a7f57150806a None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:07.641756 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e5ffa-3a7d-4cc3-a7f2-a7f57150806a None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:07.655467 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e5ffa-3a7d-4cc3-a7f2-a7f57150806a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:07.672080 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:07.691928 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-1b9e5ffa-3a7d-4cc3-a7f2-a7f57150806a None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:07.693375 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-1b9e5ffa-3a7d-4cc3-a7f2-a7f57150806a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:07.733944 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 974/1963] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:07.742254 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadecb46-d533-4021-88dd-1bbed4317f67 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:07.742539 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadecb46-d533-4021-88dd-1bbed4317f67 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:07.742813 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadecb46-d533-4021-88dd-1bbed4317f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:07.770159 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-eadecb46-d533-4021-88dd-1bbed4317f67 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:07.771677 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-eadecb46-d533-4021-88dd-1bbed4317f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:07.821539 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 990/1964] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:07.831730 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac4d09d-180e-404c-ae5a-e7b65a7276bc None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:07.832010 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac4d09d-180e-404c-ae5a-e7b65a7276bc None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:07.832443 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac4d09d-180e-404c-ae5a-e7b65a7276bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:07.852022 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 975/1965] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:07.856383 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-aac4d09d-180e-404c-ae5a-e7b65a7276bc None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:07.856516 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadecb46-d533-4021-88dd-1bbed4317f67 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:07.858583 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-aac4d09d-180e-404c-ae5a-e7b65a7276bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:07.867154 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadecb46-d533-4021-88dd-1bbed4317f67 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:07.871242 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 991/1966] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:07.872451 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac4d09d-180e-404c-ae5a-e7b65a7276bc None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:07.876006 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48735f9c-0c93-4bda-a999-c60ed33e7620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:07.877207 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48735f9c-0c93-4bda-a999-c60ed33e7620 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:07.877761 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48735f9c-0c93-4bda-a999-c60ed33e7620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:07.877981 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48735f9c-0c93-4bda-a999-c60ed33e7620 admin admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:07.878853 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48735f9c-0c93-4bda-a999-c60ed33e7620 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:43:07.879296 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48735f9c-0c93-4bda-a999-c60ed33e7620 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:07.880924 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48735f9c-0c93-4bda-a999-c60ed33e7620 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:07.882576 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac4d09d-180e-404c-ae5a-e7b65a7276bc None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:07.888158 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17129299-a0b9-4415-929b-2dfb74774559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:07.888976 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17129299-a0b9-4415-929b-2dfb74774559 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:07.889322 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17129299-a0b9-4415-929b-2dfb74774559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:07.889656 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17129299-a0b9-4415-929b-2dfb74774559 admin admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:07.890386 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17129299-a0b9-4415-929b-2dfb74774559 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:43:07.890881 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17129299-a0b9-4415-929b-2dfb74774559 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:07.892620 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17129299-a0b9-4415-929b-2dfb74774559 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:07.898218 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 976/1967] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 14:43:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:43:07.899814 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48735f9c-0c93-4bda-a999-c60ed33e7620 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:07.907811 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 992/1968] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 14:43:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:43:07.912394 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17129299-a0b9-4415-929b-2dfb74774559 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:07.914272 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48735f9c-0c93-4bda-a999-c60ed33e7620 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:07.921817 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0551293e-cb5b-4b60-9a7b-aa69d9852b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:07.922706 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551293e-cb5b-4b60-9a7b-aa69d9852b4b admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:07.923163 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551293e-cb5b-4b60-9a7b-aa69d9852b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:07.923782 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551293e-cb5b-4b60-9a7b-aa69d9852b4b admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:07.924862 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0551293e-cb5b-4b60-9a7b-aa69d9852b4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:07.924970 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17129299-a0b9-4415-929b-2dfb74774559 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:07.928407 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0551293e-cb5b-4b60-9a7b-aa69d9852b4b admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:07.929810 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10fa233-b34d-4ce6-b9f5-1df7d4c164f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:07.930423 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10fa233-b34d-4ce6-b9f5-1df7d4c164f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:07.930706 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10fa233-b34d-4ce6-b9f5-1df7d4c164f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:07.931000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10fa233-b34d-4ce6-b9f5-1df7d4c164f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:07.931659 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10fa233-b34d-4ce6-b9f5-1df7d4c164f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:07.933578 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10fa233-b34d-4ce6-b9f5-1df7d4c164f7 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:07.966333 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 977/1969] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:43:07 2022] POST /identity/v3/projects => generated 385 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:07.970404 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 993/1970] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:43:07 2022] POST /identity/v3/projects => generated 365 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:43:07.980939 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0551293e-cb5b-4b60-9a7b-aa69d9852b4b admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:07.982324 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10fa233-b34d-4ce6-b9f5-1df7d4c164f7 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:08.090056 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0551293e-cb5b-4b60-9a7b-aa69d9852b4b admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:08.094751 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:08.112922 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10fa233-b34d-4ce6-b9f5-1df7d4c164f7 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:08.122688 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4434e167-40ef-4f40-b268-f47f82292478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:08.123932 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4434e167-40ef-4f40-b268-f47f82292478 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:08.124517 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4434e167-40ef-4f40-b268-f47f82292478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:08.124865 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4434e167-40ef-4f40-b268-f47f82292478 admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:08.125613 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4434e167-40ef-4f40-b268-f47f82292478 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:08.127874 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4434e167-40ef-4f40-b268-f47f82292478 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:08.128428 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:08.161193 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 994/1971] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:43:07 2022] POST /identity/v3/users => generated 283 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:08.166373 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4434e167-40ef-4f40-b268-f47f82292478 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:08.168947 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-4434e167-40ef-4f40-b268-f47f82292478 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:08.169105 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd139d90-e093-4b6c-aa9c-880a684a7035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:08.170082 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd139d90-e093-4b6c-aa9c-880a684a7035 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:08.170790 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd139d90-e093-4b6c-aa9c-880a684a7035 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:08.171251 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd139d90-e093-4b6c-aa9c-880a684a7035 admin admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:08.183133 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd139d90-e093-4b6c-aa9c-880a684a7035 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:08.183133 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd139d90-e093-4b6c-aa9c-880a684a7035 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:08.190905 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4434e167-40ef-4f40-b268-f47f82292478 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:08.196460 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67b1fd-a588-4668-a450-832b226bfc4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:08.197300 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67b1fd-a588-4668-a450-832b226bfc4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:08.197654 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67b1fd-a588-4668-a450-832b226bfc4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:08.198022 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67b1fd-a588-4668-a450-832b226bfc4b admin admin] PATH_INFO: `/v3/projects/c2c79cd6cac141c3892a3131d462c287` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:08.199393 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de67b1fd-a588-4668-a450-832b226bfc4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2c79cd6cac141c3892a3131d462c287)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:08.202364 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de67b1fd-a588-4668-a450-832b226bfc4b admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:08.211717 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 978/1972] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:43:07 2022] POST /identity/v3/users => generated 293 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:08.211717 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd139d90-e093-4b6c-aa9c-880a684a7035 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:08.238280 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd139d90-e093-4b6c-aa9c-880a684a7035 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:08.248932 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291af3fa-a54c-423f-be63-8e90069855f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:08.249691 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291af3fa-a54c-423f-be63-8e90069855f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:08.250093 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291af3fa-a54c-423f-be63-8e90069855f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:08.250372 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291af3fa-a54c-423f-be63-8e90069855f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:08.251339 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291af3fa-a54c-423f-be63-8e90069855f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:08.253064 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291af3fa-a54c-423f-be63-8e90069855f2 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:08.266563 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 979/1973] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:08 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:08.267855 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291af3fa-a54c-423f-be63-8e90069855f2 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:08.277743 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-de67b1fd-a588-4668-a450-832b226bfc4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2c79cd6cac141c3892a3131d462c287', 'request_id': 'req-de67b1fd-a588-4668-a450-832b226bfc4b'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:08.285721 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-de67b1fd-a588-4668-a450-832b226bfc4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2c79cd6cac141c3892a3131d462c287', 'request_id': 'req-de67b1fd-a588-4668-a450-832b226bfc4b'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:08.288317 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291af3fa-a54c-423f-be63-8e90069855f2 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:08.296420 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-de67b1fd-a588-4668-a450-832b226bfc4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2c79cd6cac141c3892a3131d462c287', 'request_id': 'req-de67b1fd-a588-4668-a450-832b226bfc4b'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:08.319536 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-de67b1fd-a588-4668-a450-832b226bfc4b admin admin] The token cache is being invalidate because project c2c79cd6cac141c3892a3131d462c287 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:43:08.320006 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-de67b1fd-a588-4668-a450-832b226bfc4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:08.322059 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-de67b1fd-a588-4668-a450-832b226bfc4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:08.324241 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 995/1974] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:43:08 2022] DELETE /identity/v3/projects/c2c79cd6cac141c3892a3131d462c287 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:08.329869 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67b1fd-a588-4668-a450-832b226bfc4b admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:08.331240 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-de67b1fd-a588-4668-a450-832b226bfc4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:08.375379 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1589d30a-2e81-44a4-874e-4e7f848bbb64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:08.376337 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589d30a-2e81-44a4-874e-4e7f848bbb64 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:08.376581 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589d30a-2e81-44a4-874e-4e7f848bbb64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:08.376850 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589d30a-2e81-44a4-874e-4e7f848bbb64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:08.378146 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1589d30a-2e81-44a4-874e-4e7f848bbb64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:08.380567 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1589d30a-2e81-44a4-874e-4e7f848bbb64 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:08.396365 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 980/1975] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:08 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:08.396365 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce6522c-9360-49d6-8e9b-dd27887e6e1e None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:08.396365 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce6522c-9360-49d6-8e9b-dd27887e6e1e None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:08.396365 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce6522c-9360-49d6-8e9b-dd27887e6e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:08.417897 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-dce6522c-9360-49d6-8e9b-dd27887e6e1e None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:08.418099 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67b1fd-a588-4668-a450-832b226bfc4b admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:08.419117 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-dce6522c-9360-49d6-8e9b-dd27887e6e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:08.422658 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed92fc0c-8244-40ee-aebb-0535a18b671a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:08.423303 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed92fc0c-8244-40ee-aebb-0535a18b671a admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:08.423586 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed92fc0c-8244-40ee-aebb-0535a18b671a admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:08.423856 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed92fc0c-8244-40ee-aebb-0535a18b671a admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:08.424495 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed92fc0c-8244-40ee-aebb-0535a18b671a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:08.426080 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed92fc0c-8244-40ee-aebb-0535a18b671a admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:08.427489 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 981/1976] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:08.428281 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce6522c-9360-49d6-8e9b-dd27887e6e1e None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:08.429494 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-dce6522c-9360-49d6-8e9b-dd27887e6e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:08.436627 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 996/1977] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:08 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:08.438026 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5f027-1caf-4b23-a50a-caa49d3c1c03 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:08.438310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5f027-1caf-4b23-a50a-caa49d3c1c03 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:08.438583 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5f027-1caf-4b23-a50a-caa49d3c1c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:08.444098 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce6522c-9360-49d6-8e9b-dd27887e6e1e None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:08.448909 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02698edf-a922-4b22-9206-6e8ee686f71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:08.449504 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02698edf-a922-4b22-9206-6e8ee686f71a admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:08.449778 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02698edf-a922-4b22-9206-6e8ee686f71a admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:08.450044 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02698edf-a922-4b22-9206-6e8ee686f71a admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:08.450686 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02698edf-a922-4b22-9206-6e8ee686f71a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:08.453465 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02698edf-a922-4b22-9206-6e8ee686f71a admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:08.458887 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-afd5f027-1caf-4b23-a50a-caa49d3c1c03 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:08.460113 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-afd5f027-1caf-4b23-a50a-caa49d3c1c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:08.463679 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 982/1978] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:08.464416 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02698edf-a922-4b22-9206-6e8ee686f71a admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:08.468842 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 997/1979] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:08.469469 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd5f027-1caf-4b23-a50a-caa49d3c1c03 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:08.476965 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02698edf-a922-4b22-9206-6e8ee686f71a admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:08.478586 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd5f027-1caf-4b23-a50a-caa49d3c1c03 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:08.481662 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7263323b-b303-406d-8ff6-b5b5e6f7593c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:08.482426 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7263323b-b303-406d-8ff6-b5b5e6f7593c admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:08.482762 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7263323b-b303-406d-8ff6-b5b5e6f7593c admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:08.483100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7263323b-b303-406d-8ff6-b5b5e6f7593c admin admin] PATH_INFO: `/v3/projects/be699867b7c945c19c9d409a8133ff0c/users/c21e468aaaf5436a9d0e98cc5b5c9def/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:08.483373 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e29a48-2e27-4f77-b858-2a069f749ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:08.484017 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e29a48-2e27-4f77-b858-2a069f749ea5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:08.484302 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e29a48-2e27-4f77-b858-2a069f749ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:08.484585 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e29a48-2e27-4f77-b858-2a069f749ea5 admin admin] PATH_INFO: `/v3/projects/9d7c673313b842a9b34fd8d0258aa61f/users/916c9fe27ccb43e38a0d0abd260cfc53/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:08.499233 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7263323b-b303-406d-8ff6-b5b5e6f7593c admin admin] RBAC: Authorizing `identity:create_grant(project_id=be699867b7c945c19c9d409a8133ff0c, user_id=c21e468aaaf5436a9d0e98cc5b5c9def, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:08.499678 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e29a48-2e27-4f77-b858-2a069f749ea5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d7c673313b842a9b34fd8d0258aa61f, user_id=916c9fe27ccb43e38a0d0abd260cfc53, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:08.501999 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7263323b-b303-406d-8ff6-b5b5e6f7593c admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:08.502317 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e29a48-2e27-4f77-b858-2a069f749ea5 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:08.516232 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 983/1980] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:08 2022] PUT /identity/v3/projects/be699867b7c945c19c9d409a8133ff0c/users/c21e468aaaf5436a9d0e98cc5b5c9def/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:08.517061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 998/1981] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:08 2022] PUT /identity/v3/projects/9d7c673313b842a9b34fd8d0258aa61f/users/916c9fe27ccb43e38a0d0abd260cfc53/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:08.524109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7263323b-b303-406d-8ff6-b5b5e6f7593c admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:08.527371 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80d5e5-05de-49f0-b3ec-8bd6f73d80a4 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:08.527863 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80d5e5-05de-49f0-b3ec-8bd6f73d80a4 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:08.528365 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80d5e5-05de-49f0-b3ec-8bd6f73d80a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:08.556596 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-4f80d5e5-05de-49f0-b3ec-8bd6f73d80a4 None None] MFA Rules not processed for user `c21e468aaaf5436a9d0e98cc5b5c9def`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:08.557831 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-4f80d5e5-05de-49f0-b3ec-8bd6f73d80a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:08.578309 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:08.587244 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7263323b-b303-406d-8ff6-b5b5e6f7593c admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:08.599778 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:08.636739 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 999/1982] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:43:08.636739 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f2c32c-4718-4750-b46c-e3fd4a998159 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:08.636739 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f2c32c-4718-4750-b46c-e3fd4a998159 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:08.636739 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f2c32c-4718-4750-b46c-e3fd4a998159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:08.654165 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06652eca-edec-4886-84c2-3bb45782d096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:08.654801 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06652eca-edec-4886-84c2-3bb45782d096 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:08.655073 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06652eca-edec-4886-84c2-3bb45782d096 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:08.655387 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06652eca-edec-4886-84c2-3bb45782d096 admin admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:08.656216 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06652eca-edec-4886-84c2-3bb45782d096 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:43:08.656588 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06652eca-edec-4886-84c2-3bb45782d096 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:08.658219 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06652eca-edec-4886-84c2-3bb45782d096 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:08.666655 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-03f2c32c-4718-4750-b46c-e3fd4a998159 None None] MFA Rules not processed for user `916c9fe27ccb43e38a0d0abd260cfc53`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:08.668032 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-03f2c32c-4718-4750-b46c-e3fd4a998159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:08.674716 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 984/1983] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 14:43:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:43:08.674716 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06652eca-edec-4886-84c2-3bb45782d096 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:08.696446 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06652eca-edec-4886-84c2-3bb45782d096 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:08.701792 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc59c38-3426-47f0-bc5e-5b7b5a5e5c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:08.702404 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc59c38-3426-47f0-bc5e-5b7b5a5e5c44 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:08.702677 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc59c38-3426-47f0-bc5e-5b7b5a5e5c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:08.702940 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc59c38-3426-47f0-bc5e-5b7b5a5e5c44 admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:08.704410 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc59c38-3426-47f0-bc5e-5b7b5a5e5c44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:08.706164 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc59c38-3426-47f0-bc5e-5b7b5a5e5c44 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:08.748420 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 985/1984] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:43:08 2022] POST /identity/v3/projects => generated 365 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:08.749951 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc59c38-3426-47f0-bc5e-5b7b5a5e5c44 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:08.769352 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1000/1985] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:08.770280 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f2c32c-4718-4750-b46c-e3fd4a998159 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:08.782797 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f2c32c-4718-4750-b46c-e3fd4a998159 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:08.844352 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc59c38-3426-47f0-bc5e-5b7b5a5e5c44 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:08.855892 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdff3680-cb3d-48a7-a6a9-2cb60bf407a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:08.858030 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff3680-cb3d-48a7-a6a9-2cb60bf407a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:08.858879 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff3680-cb3d-48a7-a6a9-2cb60bf407a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:08.859164 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff3680-cb3d-48a7-a6a9-2cb60bf407a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:08.859851 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdff3680-cb3d-48a7-a6a9-2cb60bf407a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:08.862311 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdff3680-cb3d-48a7-a6a9-2cb60bf407a1 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:08.905479 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035e54da-ecd3-4d70-bda1-06901b288e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:08.907083 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e54da-ecd3-4d70-bda1-06901b288e39 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:08.910822 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e54da-ecd3-4d70-bda1-06901b288e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:08.911162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e54da-ecd3-4d70-bda1-06901b288e39 admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:08.913338 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035e54da-ecd3-4d70-bda1-06901b288e39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:08.915538 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035e54da-ecd3-4d70-bda1-06901b288e39 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:08.929269 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 986/1986] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:43:08 2022] POST /identity/v3/projects => generated 385 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:08.929269 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdff3680-cb3d-48a7-a6a9-2cb60bf407a1 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:08.932622 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-cdff3680-cb3d-48a7-a6a9-2cb60bf407a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:08.953092 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1001/1987] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:43:08 2022] POST /identity/v3/users => generated 289 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:08.953786 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035e54da-ecd3-4d70-bda1-06901b288e39 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:08.969561 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035e54da-ecd3-4d70-bda1-06901b288e39 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:09.035668 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdff3680-cb3d-48a7-a6a9-2cb60bf407a1 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:09.044291 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f34621-8f2e-482a-8e9d-471c1c175185 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:09.045603 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f34621-8f2e-482a-8e9d-471c1c175185 service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:09.045951 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f34621-8f2e-482a-8e9d-471c1c175185 service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:09.046548 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f34621-8f2e-482a-8e9d-471c1c175185 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:09.063444 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:09.093134 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:09.117135 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f986c29f-44b1-48d5-954b-361579320462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:09.117686 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986c29f-44b1-48d5-954b-361579320462 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:09.117957 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986c29f-44b1-48d5-954b-361579320462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:09.118215 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986c29f-44b1-48d5-954b-361579320462 admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:09.118860 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f986c29f-44b1-48d5-954b-361579320462 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:09.120837 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f986c29f-44b1-48d5-954b-361579320462 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:09.171027 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1002/1988] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:43:08 2022] POST /identity/v3/users => generated 293 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:09.172103 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f986c29f-44b1-48d5-954b-361579320462 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:09.173791 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f34621-8f2e-482a-8e9d-471c1c175185 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:09.176065 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f34621-8f2e-482a-8e9d-471c1c175185 service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:09.189581 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 987/1989] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:43:08 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:43:09.190802 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f34621-8f2e-482a-8e9d-471c1c175185 service cinder] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:09.203261 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f34621-8f2e-482a-8e9d-471c1c175185 service cinder] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:09.212190 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f986c29f-44b1-48d5-954b-361579320462 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:09.224561 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be838fac-ec4c-47f7-a7ac-8307aa3c0b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:09.225341 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be838fac-ec4c-47f7-a7ac-8307aa3c0b17 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:09.225580 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be838fac-ec4c-47f7-a7ac-8307aa3c0b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:09.225889 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be838fac-ec4c-47f7-a7ac-8307aa3c0b17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:09.226492 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be838fac-ec4c-47f7-a7ac-8307aa3c0b17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:09.228180 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d841bff-6788-4993-a278-ae4012532fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:09.228340 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be838fac-ec4c-47f7-a7ac-8307aa3c0b17 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:09.229702 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d841bff-6788-4993-a278-ae4012532fda admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:09.229996 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d841bff-6788-4993-a278-ae4012532fda admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:09.230274 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d841bff-6788-4993-a278-ae4012532fda admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:09.230934 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d841bff-6788-4993-a278-ae4012532fda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:09.233278 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d841bff-6788-4993-a278-ae4012532fda admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:09.241781 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 988/1990] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:09.242754 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be838fac-ec4c-47f7-a7ac-8307aa3c0b17 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:09.256261 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be838fac-ec4c-47f7-a7ac-8307aa3c0b17 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:09.264170 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecdaa2d-0068-4738-b917-86db96402993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:09.265217 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecdaa2d-0068-4738-b917-86db96402993 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:09.265776 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecdaa2d-0068-4738-b917-86db96402993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:09.266300 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecdaa2d-0068-4738-b917-86db96402993 admin admin] PATH_INFO: `/v3/users/916c9fe27ccb43e38a0d0abd260cfc53` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:09.267011 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1003/1991] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:09.268192 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecdaa2d-0068-4738-b917-86db96402993 admin admin] RBAC: Authorizing `identity:delete_user(user_id=916c9fe27ccb43e38a0d0abd260cfc53)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:09.270713 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d841bff-6788-4993-a278-ae4012532fda admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:09.271010 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecdaa2d-0068-4738-b917-86db96402993 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:09.292745 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d841bff-6788-4993-a278-ae4012532fda admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:09.305221 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12bb85b-2072-42c9-ae33-e7b741d1257d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:09.305857 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12bb85b-2072-42c9-ae33-e7b741d1257d admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:09.306146 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12bb85b-2072-42c9-ae33-e7b741d1257d admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:09.306424 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12bb85b-2072-42c9-ae33-e7b741d1257d admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:09.307125 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12bb85b-2072-42c9-ae33-e7b741d1257d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:09.308975 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12bb85b-2072-42c9-ae33-e7b741d1257d admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:09.328427 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1004/1992] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:09.329448 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12bb85b-2072-42c9-ae33-e7b741d1257d admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:09.343286 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12bb85b-2072-42c9-ae33-e7b741d1257d admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:09.359682 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-9ecdaa2d-0068-4738-b917-86db96402993 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '916c9fe27ccb43e38a0d0abd260cfc53'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:09.360096 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aa9c5d-3015-4352-8605-0b06781ac3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:09.360226 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-9ecdaa2d-0068-4738-b917-86db96402993 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '916c9fe27ccb43e38a0d0abd260cfc53'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:09.360816 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa9c5d-3015-4352-8605-0b06781ac3aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:09.361067 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa9c5d-3015-4352-8605-0b06781ac3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:09.361438 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa9c5d-3015-4352-8605-0b06781ac3aa admin admin] PATH_INFO: `/v3/projects/f1b728aa400d47fca5218f9e8b693959/users/6a2da2d31f4946d3b5d94d5af886991c/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:09.375683 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-9ecdaa2d-0068-4738-b917-86db96402993 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '916c9fe27ccb43e38a0d0abd260cfc53'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:09.386679 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-9ecdaa2d-0068-4738-b917-86db96402993 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '916c9fe27ccb43e38a0d0abd260cfc53'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:09.392356 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aa9c5d-3015-4352-8605-0b06781ac3aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1b728aa400d47fca5218f9e8b693959, user_id=6a2da2d31f4946d3b5d94d5af886991c, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:09.395060 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aa9c5d-3015-4352-8605-0b06781ac3aa admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:09.408565 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-9ecdaa2d-0068-4738-b917-86db96402993 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '916c9fe27ccb43e38a0d0abd260cfc53'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:09.417101 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1005/1993] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:09 2022] PUT /identity/v3/projects/f1b728aa400d47fca5218f9e8b693959/users/6a2da2d31f4946d3b5d94d5af886991c/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:09.417101 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aa9c5d-3015-4352-8605-0b06781ac3aa admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:09.422590 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-38aa9c5d-3015-4352-8605-0b06781ac3aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:09.422767 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 989/1994] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:43:09 2022] DELETE /identity/v3/users/916c9fe27ccb43e38a0d0abd260cfc53 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:09.422767 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecdaa2d-0068-4738-b917-86db96402993 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:09.422767 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-9ecdaa2d-0068-4738-b917-86db96402993 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:09.535459 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecdaa2d-0068-4738-b917-86db96402993 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:09.551441 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca342ca1-5772-4371-8872-acde9ae2fc9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:09.552119 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca342ca1-5772-4371-8872-acde9ae2fc9d admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:09.552404 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca342ca1-5772-4371-8872-acde9ae2fc9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:09.552708 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca342ca1-5772-4371-8872-acde9ae2fc9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:09.554489 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aa9c5d-3015-4352-8605-0b06781ac3aa admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:09.555710 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca342ca1-5772-4371-8872-acde9ae2fc9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:09.558891 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca342ca1-5772-4371-8872-acde9ae2fc9d admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:09.568843 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7b7146-cd62-4d6d-a6cb-0e622ca1ef12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:09.569517 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b7146-cd62-4d6d-a6cb-0e622ca1ef12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:09.569838 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b7146-cd62-4d6d-a6cb-0e622ca1ef12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:09.570115 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b7146-cd62-4d6d-a6cb-0e622ca1ef12 admin admin] PATH_INFO: `/v3/projects/9f437fc48ab748c0bec905e55642deaa/users/bd0fb5e4286842e8acc424eb4163c2cd/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:09.586784 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 990/1995] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:09.596408 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea9f6a-cd67-41f7-9359-f2fd49c59bfb None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:09.596706 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea9f6a-cd67-41f7-9359-f2fd49c59bfb None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:09.597069 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea9f6a-cd67-41f7-9359-f2fd49c59bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:09.613554 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7b7146-cd62-4d6d-a6cb-0e622ca1ef12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f437fc48ab748c0bec905e55642deaa, user_id=bd0fb5e4286842e8acc424eb4163c2cd, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:09.620139 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7b7146-cd62-4d6d-a6cb-0e622ca1ef12 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:09.652219 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-9dea9f6a-cd67-41f7-9359-f2fd49c59bfb None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:09.653973 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-9dea9f6a-cd67-41f7-9359-f2fd49c59bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:09.668152 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1006/1996] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:09 2022] PUT /identity/v3/projects/9f437fc48ab748c0bec905e55642deaa/users/bd0fb5e4286842e8acc424eb4163c2cd/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:09.669332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7b7146-cd62-4d6d-a6cb-0e622ca1ef12 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:09.704612 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:09.780644 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 991/1997] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:09.786938 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7b7146-cd62-4d6d-a6cb-0e622ca1ef12 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:09.787350 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40f8a4-36c1-4400-b15c-8f608e31df21 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:09.787844 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40f8a4-36c1-4400-b15c-8f608e31df21 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:09.788318 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40f8a4-36c1-4400-b15c-8f608e31df21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:09.805516 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aad77e-f543-4556-b6a0-79bcfc3ea149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:09.806208 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aad77e-f543-4556-b6a0-79bcfc3ea149 admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:09.806500 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aad77e-f543-4556-b6a0-79bcfc3ea149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:09.806791 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aad77e-f543-4556-b6a0-79bcfc3ea149 admin admin] PATH_INFO: `/v3/projects/f1b728aa400d47fca5218f9e8b693959/users/6a2da2d31f4946d3b5d94d5af886991c/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:09.812807 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20aad77e-f543-4556-b6a0-79bcfc3ea149 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1b728aa400d47fca5218f9e8b693959, user_id=6a2da2d31f4946d3b5d94d5af886991c, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:09.818604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20aad77e-f543-4556-b6a0-79bcfc3ea149 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:09.830820 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-ad40f8a4-36c1-4400-b15c-8f608e31df21 None None] MFA Rules not processed for user `bd0fb5e4286842e8acc424eb4163c2cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:09.832331 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-ad40f8a4-36c1-4400-b15c-8f608e31df21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:09.835297 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:09.844756 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1007/1998] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:09 2022] PUT /identity/v3/projects/f1b728aa400d47fca5218f9e8b693959/users/6a2da2d31f4946d3b5d94d5af886991c/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:09.847375 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aad77e-f543-4556-b6a0-79bcfc3ea149 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:09.848097 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-20aad77e-f543-4556-b6a0-79bcfc3ea149 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:09.930113 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aad77e-f543-4556-b6a0-79bcfc3ea149 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:09.933417 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 992/1999] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:09.933989 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14500b11-894b-4831-aa31-17a77d2d400d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:09.934601 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14500b11-894b-4831-aa31-17a77d2d400d service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:09.934885 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14500b11-894b-4831-aa31-17a77d2d400d service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:09.935160 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14500b11-894b-4831-aa31-17a77d2d400d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:09.938622 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4adda8-b5ec-4846-81b1-a869e648e032 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:09.939013 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4adda8-b5ec-4846-81b1-a869e648e032 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:09.939325 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4adda8-b5ec-4846-81b1-a869e648e032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:09.987237 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-cf4adda8-b5ec-4846-81b1-a869e648e032 None None] MFA Rules not processed for user `6a2da2d31f4946d3b5d94d5af886991c`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:09.988496 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-cf4adda8-b5ec-4846-81b1-a869e648e032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:10.005396 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14500b11-894b-4831-aa31-17a77d2d400d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:10.006842 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14500b11-894b-4831-aa31-17a77d2d400d service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:10.017255 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1008/2000] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:43:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:43:10.018062 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14500b11-894b-4831-aa31-17a77d2d400d service neutron] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:10.028477 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14500b11-894b-4831-aa31-17a77d2d400d service neutron] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:10.034025 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad91b4-ac6c-408d-b59a-a1cfdef1dd02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:10.034761 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad91b4-ac6c-408d-b59a-a1cfdef1dd02 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:10.035035 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad91b4-ac6c-408d-b59a-a1cfdef1dd02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:10.035345 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad91b4-ac6c-408d-b59a-a1cfdef1dd02 admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:10.035984 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ad91b4-ac6c-408d-b59a-a1cfdef1dd02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:10.037927 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ad91b4-ac6c-408d-b59a-a1cfdef1dd02 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:10.071253 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 993/2001] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:09 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:10.077944 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1009/2002] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:43:10 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:10.081044 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4adda8-b5ec-4846-81b1-a869e648e032 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:10.081820 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad91b4-ac6c-408d-b59a-a1cfdef1dd02 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:10.083257 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-cf4adda8-b5ec-4846-81b1-a869e648e032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:10.133126 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:10.146925 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4adda8-b5ec-4846-81b1-a869e648e032 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:10.153356 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad91b4-ac6c-408d-b59a-a1cfdef1dd02 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:10.159179 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d399abe8-a633-4ee1-b825-23934ef97334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:10.159817 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399abe8-a633-4ee1-b825-23934ef97334 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:10.160106 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399abe8-a633-4ee1-b825-23934ef97334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:10.160387 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399abe8-a633-4ee1-b825-23934ef97334 admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:10.161186 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d399abe8-a633-4ee1-b825-23934ef97334 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:10.163334 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d399abe8-a633-4ee1-b825-23934ef97334 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:10.192467 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1010/2003] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:43:10 2022] POST /identity/v3/users => generated 293 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:10.196538 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d399abe8-a633-4ee1-b825-23934ef97334 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:10.206282 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7c1aca-208e-455a-b3c5-cf53b2a1ddb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:10.207002 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c1aca-208e-455a-b3c5-cf53b2a1ddb3 service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:10.207460 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c1aca-208e-455a-b3c5-cf53b2a1ddb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:10.207883 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c1aca-208e-455a-b3c5-cf53b2a1ddb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:10.209958 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d399abe8-a633-4ee1-b825-23934ef97334 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:10.217675 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d1dced-a843-499c-9680-e39b6f824800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:10.219264 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1dced-a843-499c-9680-e39b6f824800 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:10.219264 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1dced-a843-499c-9680-e39b6f824800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:10.219264 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1dced-a843-499c-9680-e39b6f824800 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:10.219696 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d1dced-a843-499c-9680-e39b6f824800 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:10.222197 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d1dced-a843-499c-9680-e39b6f824800 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:10.238576 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1011/2004] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:10 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:10.239859 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d1dced-a843-499c-9680-e39b6f824800 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:10.253958 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d1dced-a843-499c-9680-e39b6f824800 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:10.258696 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24dad2-e377-4973-b87f-11be12baf015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:10.259349 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24dad2-e377-4973-b87f-11be12baf015 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:10.259613 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24dad2-e377-4973-b87f-11be12baf015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:10.259885 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24dad2-e377-4973-b87f-11be12baf015 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:10.260550 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e24dad2-e377-4973-b87f-11be12baf015 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:10.262135 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e24dad2-e377-4973-b87f-11be12baf015 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:10.264106 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:10.276239 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1012/2005] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:10.280084 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24dad2-e377-4973-b87f-11be12baf015 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:10.293055 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24dad2-e377-4973-b87f-11be12baf015 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:10.298707 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7c1aca-208e-455a-b3c5-cf53b2a1ddb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:10.299798 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c4e0a8-4530-42e5-b02c-b15cd3671858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:10.300553 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4e0a8-4530-42e5-b02c-b15cd3671858 admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:10.300669 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7c1aca-208e-455a-b3c5-cf53b2a1ddb3 service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:10.300964 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4e0a8-4530-42e5-b02c-b15cd3671858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:10.301310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4e0a8-4530-42e5-b02c-b15cd3671858 admin admin] PATH_INFO: `/v3/projects/dda573e027aa4d90bca096c157d14fdc/users/9130b4bf347b402ca2070f52b4b5b9c4/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:10.312692 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 994/2006] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:43:10 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:43:10.323505 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c4e0a8-4530-42e5-b02c-b15cd3671858 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dda573e027aa4d90bca096c157d14fdc, user_id=9130b4bf347b402ca2070f52b4b5b9c4, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:10.327684 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c4e0a8-4530-42e5-b02c-b15cd3671858 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:10.345881 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7c1aca-208e-455a-b3c5-cf53b2a1ddb3 service cinder] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:10.348556 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-bf7c1aca-208e-455a-b3c5-cf53b2a1ddb3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:10.349429 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1013/2007] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:10 2022] PUT /identity/v3/projects/dda573e027aa4d90bca096c157d14fdc/users/9130b4bf347b402ca2070f52b4b5b9c4/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:43:10.362255 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5011f68-ba26-419c-b33b-8b6b6e4ddf98 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:10.363677 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5011f68-ba26-419c-b33b-8b6b6e4ddf98 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:10.364744 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5011f68-ba26-419c-b33b-8b6b6e4ddf98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:10.400548 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7c1aca-208e-455a-b3c5-cf53b2a1ddb3 service cinder] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:10.416822 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-f5011f68-ba26-419c-b33b-8b6b6e4ddf98 None None] MFA Rules not processed for user `9130b4bf347b402ca2070f52b4b5b9c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:10.418355 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-f5011f68-ba26-419c-b33b-8b6b6e4ddf98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:10.470582 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b87ac-e7a9-4696-b331-703ae4a3a35a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:10.471228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b87ac-e7a9-4696-b331-703ae4a3a35a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:10.471521 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b87ac-e7a9-4696-b331-703ae4a3a35a admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:10.471799 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b87ac-e7a9-4696-b331-703ae4a3a35a admin admin] PATH_INFO: `/v3/projects/9d7c673313b842a9b34fd8d0258aa61f` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:10.472965 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700b87ac-e7a9-4696-b331-703ae4a3a35a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d7c673313b842a9b34fd8d0258aa61f)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:10.474879 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700b87ac-e7a9-4696-b331-703ae4a3a35a admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:10.489202 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1014/2008] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:43:10.490127 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5011f68-ba26-419c-b33b-8b6b6e4ddf98 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:10.524791 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-700b87ac-e7a9-4696-b331-703ae4a3a35a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d7c673313b842a9b34fd8d0258aa61f', 'request_id': 'req-700b87ac-e7a9-4696-b331-703ae4a3a35a'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:10.525976 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:10.532206 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-700b87ac-e7a9-4696-b331-703ae4a3a35a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d7c673313b842a9b34fd8d0258aa61f', 'request_id': 'req-700b87ac-e7a9-4696-b331-703ae4a3a35a'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:10.539065 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-700b87ac-e7a9-4696-b331-703ae4a3a35a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d7c673313b842a9b34fd8d0258aa61f', 'request_id': 'req-700b87ac-e7a9-4696-b331-703ae4a3a35a'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:10.544221 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5011f68-ba26-419c-b33b-8b6b6e4ddf98 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:10.553281 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-700b87ac-e7a9-4696-b331-703ae4a3a35a admin admin] The token cache is being invalidate because project 9d7c673313b842a9b34fd8d0258aa61f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:43:10.553579 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-700b87ac-e7a9-4696-b331-703ae4a3a35a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:10.554316 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-700b87ac-e7a9-4696-b331-703ae4a3a35a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:10.556210 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 995/2009] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:43:10 2022] DELETE /identity/v3/projects/9d7c673313b842a9b34fd8d0258aa61f => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:10.556734 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b87ac-e7a9-4696-b331-703ae4a3a35a admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:10.558110 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-700b87ac-e7a9-4696-b331-703ae4a3a35a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:10.604474 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcb1a88-ed1c-4f92-a89b-aab69ebb4c12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:10.605091 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb1a88-ed1c-4f92-a89b-aab69ebb4c12 service cinder] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:10.605373 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb1a88-ed1c-4f92-a89b-aab69ebb4c12 service cinder] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:10.605641 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb1a88-ed1c-4f92-a89b-aab69ebb4c12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:10.608029 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-1dcb1a88-ed1c-4f92-a89b-aab69ebb4c12 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:10.630432 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b87ac-e7a9-4696-b331-703ae4a3a35a admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:10.637458 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5fe57b-5a25-4f32-8229-b1356186f650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:10.638074 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5fe57b-5a25-4f32-8229-b1356186f650 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:10.638353 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5fe57b-5a25-4f32-8229-b1356186f650 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:10.638666 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5fe57b-5a25-4f32-8229-b1356186f650 admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:10.639737 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5fe57b-5a25-4f32-8229-b1356186f650 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:10.641668 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5fe57b-5a25-4f32-8229-b1356186f650 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:10.645844 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:10.678946 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcb1a88-ed1c-4f92-a89b-aab69ebb4c12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:10.681223 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcb1a88-ed1c-4f92-a89b-aab69ebb4c12 service cinder] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:10.687107 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 996/2010] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:43:10 2022] POST /identity/v3/projects => generated 385 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:10.688663 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e0b988-c061-45d8-8d36-975befaadf81 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:10.689232 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e0b988-c061-45d8-8d36-975befaadf81 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:10.689349 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e0b988-c061-45d8-8d36-975befaadf81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:10.697289 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1015/2011] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:43:10 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:43:10.698172 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcb1a88-ed1c-4f92-a89b-aab69ebb4c12 service cinder] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:10.709648 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcb1a88-ed1c-4f92-a89b-aab69ebb4c12 service cinder] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:10.731480 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-09e0b988-c061-45d8-8d36-975befaadf81 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:10.733726 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-09e0b988-c061-45d8-8d36-975befaadf81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:10.787994 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469df141-62dc-4be9-8b64-7abd16578ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:10.788618 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469df141-62dc-4be9-8b64-7abd16578ae5 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:10.788905 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469df141-62dc-4be9-8b64-7abd16578ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:10.789167 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469df141-62dc-4be9-8b64-7abd16578ae5 admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:10.789805 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469df141-62dc-4be9-8b64-7abd16578ae5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:10.792085 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469df141-62dc-4be9-8b64-7abd16578ae5 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:10.819410 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1016/2012] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:43:10 2022] POST /identity/v3/users => generated 299 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:10.822430 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469df141-62dc-4be9-8b64-7abd16578ae5 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:10.827227 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 997/2013] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:10.835001 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c9a9f-dfa5-4504-abe0-965d6e774621 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:10.835442 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c9a9f-dfa5-4504-abe0-965d6e774621 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:10.836224 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c9a9f-dfa5-4504-abe0-965d6e774621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:10.843446 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469df141-62dc-4be9-8b64-7abd16578ae5 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:10.865043 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-913c9a9f-dfa5-4504-abe0-965d6e774621 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:10.866245 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b734b7-e45b-40d3-82fe-be800e3f3c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:10.866837 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-913c9a9f-dfa5-4504-abe0-965d6e774621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:10.866943 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b734b7-e45b-40d3-82fe-be800e3f3c2b admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:10.867780 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b734b7-e45b-40d3-82fe-be800e3f3c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:10.868077 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b734b7-e45b-40d3-82fe-be800e3f3c2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:10.869173 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b734b7-e45b-40d3-82fe-be800e3f3c2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:10.870968 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b734b7-e45b-40d3-82fe-be800e3f3c2b admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:10.877288 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 998/2014] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:43:10.881939 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c9a9f-dfa5-4504-abe0-965d6e774621 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:10.892262 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1017/2015] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:10 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:43:10.897375 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b734b7-e45b-40d3-82fe-be800e3f3c2b admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:10.902426 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c9a9f-dfa5-4504-abe0-965d6e774621 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:10.908245 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b734b7-e45b-40d3-82fe-be800e3f3c2b admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:10.912896 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8428e-8f46-40ab-a692-344cc448efe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:10.913653 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8428e-8f46-40ab-a692-344cc448efe3 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:10.914233 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a453696c-a7a8-4e43-be4f-35b3d088aba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:10.916017 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453696c-a7a8-4e43-be4f-35b3d088aba9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:10.916307 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453696c-a7a8-4e43-be4f-35b3d088aba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:10.916588 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453696c-a7a8-4e43-be4f-35b3d088aba9 admin admin] PATH_INFO: `/v3/projects/144e2687b09a4449a0583c3bab7362b3/users/bbea3303bbab4a1dad81339d5b446ba4/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:10.917701 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8428e-8f46-40ab-a692-344cc448efe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:10.918053 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8428e-8f46-40ab-a692-344cc448efe3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:10.918847 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f8428e-8f46-40ab-a692-344cc448efe3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:43:10.919339 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f8428e-8f46-40ab-a692-344cc448efe3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:10.921172 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f8428e-8f46-40ab-a692-344cc448efe3 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:10.934431 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 999/2016] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 14:43:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:43:10.939699 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8428e-8f46-40ab-a692-344cc448efe3 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:10.942222 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a453696c-a7a8-4e43-be4f-35b3d088aba9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=144e2687b09a4449a0583c3bab7362b3, user_id=bbea3303bbab4a1dad81339d5b446ba4, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:10.944997 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a453696c-a7a8-4e43-be4f-35b3d088aba9 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:10.951840 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8428e-8f46-40ab-a692-344cc448efe3 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:10.957969 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400b4555-161a-4082-9c48-c3241f2af5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:10.958682 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400b4555-161a-4082-9c48-c3241f2af5c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:10.959036 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400b4555-161a-4082-9c48-c3241f2af5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:10.959424 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400b4555-161a-4082-9c48-c3241f2af5c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:10.960148 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400b4555-161a-4082-9c48-c3241f2af5c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:10.964424 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400b4555-161a-4082-9c48-c3241f2af5c5 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:10.965283 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1018/2017] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:10 2022] PUT /identity/v3/projects/144e2687b09a4449a0583c3bab7362b3/users/bbea3303bbab4a1dad81339d5b446ba4/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:10.971714 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a453696c-a7a8-4e43-be4f-35b3d088aba9 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:11.013735 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1000/2018] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:43:10 2022] POST /identity/v3/projects => generated 375 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:43:11.014898 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400b4555-161a-4082-9c48-c3241f2af5c5 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:11.056023 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:11.083785 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a453696c-a7a8-4e43-be4f-35b3d088aba9 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:11.115483 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400b4555-161a-4082-9c48-c3241f2af5c5 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:11.124300 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2259ff18-1218-4970-b716-91a54bd99774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:11.125265 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2259ff18-1218-4970-b716-91a54bd99774 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:11.125552 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2259ff18-1218-4970-b716-91a54bd99774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:11.125830 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2259ff18-1218-4970-b716-91a54bd99774 admin admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:11.126044 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:11.126469 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2259ff18-1218-4970-b716-91a54bd99774 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:11.128790 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2259ff18-1218-4970-b716-91a54bd99774 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:11.151256 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1001/2019] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:43:11 2022] POST /identity/v3/users => generated 288 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:11.153714 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a52edb7-dc67-41a0-be6c-17dec3b1f922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:11.154888 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52edb7-dc67-41a0-be6c-17dec3b1f922 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:11.155166 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52edb7-dc67-41a0-be6c-17dec3b1f922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:11.155581 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52edb7-dc67-41a0-be6c-17dec3b1f922 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:11.156006 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2259ff18-1218-4970-b716-91a54bd99774 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:11.156242 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a52edb7-dc67-41a0-be6c-17dec3b1f922 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:11.157953 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a52edb7-dc67-41a0-be6c-17dec3b1f922 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:11.165688 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2259ff18-1218-4970-b716-91a54bd99774 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:11.171363 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1019/2020] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:10 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:43:11.171515 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a808c-ee4e-4846-bade-7d961202fe0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:11.171515 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a808c-ee4e-4846-bade-7d961202fe0d admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:11.171515 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a808c-ee4e-4846-bade-7d961202fe0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:11.171768 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a808c-ee4e-4846-bade-7d961202fe0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:11.172436 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401a808c-ee4e-4846-bade-7d961202fe0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:11.172813 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a52edb7-dc67-41a0-be6c-17dec3b1f922 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:11.174034 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401a808c-ee4e-4846-bade-7d961202fe0d admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:11.182849 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a52edb7-dc67-41a0-be6c-17dec3b1f922 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:11.184333 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1002/2021] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:43:11.188286 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a808c-ee4e-4846-bade-7d961202fe0d admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:11.191298 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e26ccd4-0fd6-4c46-99aa-679a73f9dffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:11.191955 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e26ccd4-0fd6-4c46-99aa-679a73f9dffe admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:11.192234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e26ccd4-0fd6-4c46-99aa-679a73f9dffe admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:11.192509 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e26ccd4-0fd6-4c46-99aa-679a73f9dffe admin admin] PATH_INFO: `/v3/projects/144e2687b09a4449a0583c3bab7362b3/users/bbea3303bbab4a1dad81339d5b446ba4/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:11.194530 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e26ccd4-0fd6-4c46-99aa-679a73f9dffe admin admin] RBAC: Authorizing `identity:create_grant(project_id=144e2687b09a4449a0583c3bab7362b3, user_id=bbea3303bbab4a1dad81339d5b446ba4, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:11.197241 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e26ccd4-0fd6-4c46-99aa-679a73f9dffe admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:11.198618 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a808c-ee4e-4846-bade-7d961202fe0d admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:11.204211 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75a959a-0966-4b30-9b79-ac60cb79167e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:11.204794 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a959a-0966-4b30-9b79-ac60cb79167e admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:11.205073 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a959a-0966-4b30-9b79-ac60cb79167e admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:11.205350 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a959a-0966-4b30-9b79-ac60cb79167e admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:11.205987 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75a959a-0966-4b30-9b79-ac60cb79167e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:11.207607 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75a959a-0966-4b30-9b79-ac60cb79167e admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:11.211286 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1020/2022] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:11 2022] PUT /identity/v3/projects/144e2687b09a4449a0583c3bab7362b3/users/bbea3303bbab4a1dad81339d5b446ba4/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:11.222345 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4a4c7-ce4b-4a62-a510-9717cd2f0816 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:11.222631 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4a4c7-ce4b-4a62-a510-9717cd2f0816 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:11.222931 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4a4c7-ce4b-4a62-a510-9717cd2f0816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:11.228200 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1003/2023] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:11 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:43:11.232058 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75a959a-0966-4b30-9b79-ac60cb79167e admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:11.242374 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75a959a-0966-4b30-9b79-ac60cb79167e admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:11.253907 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-f3b4a4c7-ce4b-4a62-a510-9717cd2f0816 None None] MFA Rules not processed for user `bbea3303bbab4a1dad81339d5b446ba4`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:11.255125 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-f3b4a4c7-ce4b-4a62-a510-9717cd2f0816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:11.301355 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9c0143-c4ce-41b1-8e3a-fbbc7a2f7a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:11.302024 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c0143-c4ce-41b1-8e3a-fbbc7a2f7a1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:11.302313 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c0143-c4ce-41b1-8e3a-fbbc7a2f7a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:11.302587 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c0143-c4ce-41b1-8e3a-fbbc7a2f7a1c admin admin] PATH_INFO: `/v3/projects/e1035b00467b43fcb44f7bbf04935650/users/b309b1abd7604db5acf80c4cfbf96982/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:11.317350 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9c0143-c4ce-41b1-8e3a-fbbc7a2f7a1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1035b00467b43fcb44f7bbf04935650, user_id=b309b1abd7604db5acf80c4cfbf96982, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:11.320035 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9c0143-c4ce-41b1-8e3a-fbbc7a2f7a1c admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:11.327768 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1021/2024] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:11 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:43:11.334135 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1004/2025] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:11 2022] PUT /identity/v3/projects/e1035b00467b43fcb44f7bbf04935650/users/b309b1abd7604db5acf80c4cfbf96982/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:43:11.339045 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b4a4c7-ce4b-4a62-a510-9717cd2f0816 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:11.340493 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-f3b4a4c7-ce4b-4a62-a510-9717cd2f0816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:11.345449 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ec53b-d8c4-4cfe-9d6e-6915c49acd36 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:11.345732 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ec53b-d8c4-4cfe-9d6e-6915c49acd36 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:11.346008 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ec53b-d8c4-4cfe-9d6e-6915c49acd36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:11.372993 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-e88ec53b-d8c4-4cfe-9d6e-6915c49acd36 None None] MFA Rules not processed for user `b309b1abd7604db5acf80c4cfbf96982`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:11.374290 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-e88ec53b-d8c4-4cfe-9d6e-6915c49acd36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:11.398936 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b4a4c7-ce4b-4a62-a510-9717cd2f0816 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:11.442974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1005/2026] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:43:11.455328 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88ec53b-d8c4-4cfe-9d6e-6915c49acd36 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:11.456441 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5194641-40e9-4c43-8e7a-43ed010e0d4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:11.457065 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-e88ec53b-d8c4-4cfe-9d6e-6915c49acd36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:11.457169 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5194641-40e9-4c43-8e7a-43ed010e0d4b service cinder] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:11.457347 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5194641-40e9-4c43-8e7a-43ed010e0d4b service cinder] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:11.457622 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5194641-40e9-4c43-8e7a-43ed010e0d4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:11.459526 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-f5194641-40e9-4c43-8e7a-43ed010e0d4b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:11.463870 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:11.470228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88ec53b-d8c4-4cfe-9d6e-6915c49acd36 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:11.471121 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:11.475240 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f396e8-9f4a-4581-9b17-595e42655cd2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:11.475670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f396e8-9f4a-4581-9b17-595e42655cd2 service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:11.475924 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f396e8-9f4a-4581-9b17-595e42655cd2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:11.476223 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f396e8-9f4a-4581-9b17-595e42655cd2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:11.484783 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f396e8-9f4a-4581-9b17-595e42655cd2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:11.486202 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f396e8-9f4a-4581-9b17-595e42655cd2 service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:11.495101 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1006/2027] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:43:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:43:11.520926 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5194641-40e9-4c43-8e7a-43ed010e0d4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:11.522430 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5194641-40e9-4c43-8e7a-43ed010e0d4b service cinder] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:11.532871 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1022/2028] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:43:11 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:43:14.635713 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f396e8-9f4a-4581-9b17-595e42655cd2 service cinder] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:14.646248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f396e8-9f4a-4581-9b17-595e42655cd2 service cinder] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:14.651975 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3e192b-28e8-47c2-97a6-6b595989e39f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:14.653026 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3e192b-28e8-47c2-97a6-6b595989e39f service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:14.653533 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3e192b-28e8-47c2-97a6-6b595989e39f service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:14.655237 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3e192b-28e8-47c2-97a6-6b595989e39f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:14.657676 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-7a3e192b-28e8-47c2-97a6-6b595989e39f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:14.727830 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3e192b-28e8-47c2-97a6-6b595989e39f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:14.730910 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3e192b-28e8-47c2-97a6-6b595989e39f service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:14.740046 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1007/2029] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:43:14 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:43:17.575264 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5194641-40e9-4c43-8e7a-43ed010e0d4b service cinder] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:17.584185 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5194641-40e9-4c43-8e7a-43ed010e0d4b service cinder] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:17.587916 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ef877-d630-4f32-95a8-88f32c64ce4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:17.588517 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ef877-d630-4f32-95a8-88f32c64ce4c service cinder] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:17.588794 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ef877-d630-4f32-95a8-88f32c64ce4c service cinder] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:17.589066 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ef877-d630-4f32-95a8-88f32c64ce4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:17.658427 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9ef877-d630-4f32-95a8-88f32c64ce4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:17.659864 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9ef877-d630-4f32-95a8-88f32c64ce4c service cinder] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:17.671467 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1023/2030] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:43:17 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:43:18.251549 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3e192b-28e8-47c2-97a6-6b595989e39f service cinder] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:18.253077 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-7a3e192b-28e8-47c2-97a6-6b595989e39f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:18.389674 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3e192b-28e8-47c2-97a6-6b595989e39f service cinder] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:18.401675 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d3c89-6f96-477b-a1ae-f9a51ce72166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:18.405640 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d3c89-6f96-477b-a1ae-f9a51ce72166 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:18.406057 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d3c89-6f96-477b-a1ae-f9a51ce72166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:18.409180 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d3c89-6f96-477b-a1ae-f9a51ce72166 admin admin] PATH_INFO: `/v3/users/c21e468aaaf5436a9d0e98cc5b5c9def` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:18.411118 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0d3c89-6f96-477b-a1ae-f9a51ce72166 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c21e468aaaf5436a9d0e98cc5b5c9def)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:18.423794 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0d3c89-6f96-477b-a1ae-f9a51ce72166 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:18.512171 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-ff0d3c89-6f96-477b-a1ae-f9a51ce72166 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c21e468aaaf5436a9d0e98cc5b5c9def'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:18.512993 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-ff0d3c89-6f96-477b-a1ae-f9a51ce72166 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c21e468aaaf5436a9d0e98cc5b5c9def'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:18.530769 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-ff0d3c89-6f96-477b-a1ae-f9a51ce72166 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c21e468aaaf5436a9d0e98cc5b5c9def'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:18.541721 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-ff0d3c89-6f96-477b-a1ae-f9a51ce72166 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c21e468aaaf5436a9d0e98cc5b5c9def'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:18.554042 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-ff0d3c89-6f96-477b-a1ae-f9a51ce72166 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c21e468aaaf5436a9d0e98cc5b5c9def'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:18.568760 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1008/2031] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:43:18 2022] DELETE /identity/v3/users/c21e468aaaf5436a9d0e98cc5b5c9def => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:18.571280 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860d6cb-1048-44c8-b992-18fede9d69a3 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:18.571576 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860d6cb-1048-44c8-b992-18fede9d69a3 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:18.572088 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860d6cb-1048-44c8-b992-18fede9d69a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:18.612373 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-f860d6cb-1048-44c8-b992-18fede9d69a3 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:18.614094 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-f860d6cb-1048-44c8-b992-18fede9d69a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:18.716003 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1024/2032] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:18.726553 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d3c89-6f96-477b-a1ae-f9a51ce72166 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:18.728596 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-ff0d3c89-6f96-477b-a1ae-f9a51ce72166 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:18.745972 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:18.807278 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d3c89-6f96-477b-a1ae-f9a51ce72166 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:18.811536 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad73c8-82be-46a6-90ab-125cdb114349 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:18.812342 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad73c8-82be-46a6-90ab-125cdb114349 service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:18.812723 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad73c8-82be-46a6-90ab-125cdb114349 service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:18.813257 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad73c8-82be-46a6-90ab-125cdb114349 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:18.825250 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ad73c8-82be-46a6-90ab-125cdb114349 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:18.827499 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ad73c8-82be-46a6-90ab-125cdb114349 service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:18.840308 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1009/2033] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:43:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:43:19.180698 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f860d6cb-1048-44c8-b992-18fede9d69a3 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:19.183374 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-f860d6cb-1048-44c8-b992-18fede9d69a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:19.208611 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f860d6cb-1048-44c8-b992-18fede9d69a3 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:19.221003 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850e2d90-f331-4f2c-b8e2-88a934e67889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:19.221798 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e2d90-f331-4f2c-b8e2-88a934e67889 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:19.222081 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e2d90-f331-4f2c-b8e2-88a934e67889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:19.222371 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e2d90-f331-4f2c-b8e2-88a934e67889 admin admin] PATH_INFO: `/v3/projects/be699867b7c945c19c9d409a8133ff0c` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:19.223780 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850e2d90-f331-4f2c-b8e2-88a934e67889 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be699867b7c945c19c9d409a8133ff0c)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:19.225681 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850e2d90-f331-4f2c-b8e2-88a934e67889 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:19.274986 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:19.284677 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-850e2d90-f331-4f2c-b8e2-88a934e67889 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be699867b7c945c19c9d409a8133ff0c', 'request_id': 'req-850e2d90-f331-4f2c-b8e2-88a934e67889'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:19.293364 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-850e2d90-f331-4f2c-b8e2-88a934e67889 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be699867b7c945c19c9d409a8133ff0c', 'request_id': 'req-850e2d90-f331-4f2c-b8e2-88a934e67889'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:19.301208 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-850e2d90-f331-4f2c-b8e2-88a934e67889 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be699867b7c945c19c9d409a8133ff0c', 'request_id': 'req-850e2d90-f331-4f2c-b8e2-88a934e67889'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:19.314478 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-850e2d90-f331-4f2c-b8e2-88a934e67889 admin admin] The token cache is being invalidate because project be699867b7c945c19c9d409a8133ff0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:43:19.314774 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-850e2d90-f331-4f2c-b8e2-88a934e67889 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:19.315711 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-850e2d90-f331-4f2c-b8e2-88a934e67889 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:19.317249 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1025/2034] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:43:19 2022] DELETE /identity/v3/projects/be699867b7c945c19c9d409a8133ff0c => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:19.321243 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad73c8-82be-46a6-90ab-125cdb114349 service neutron] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:19.331797 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad73c8-82be-46a6-90ab-125cdb114349 service neutron] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:19.387779 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850e2d90-f331-4f2c-b8e2-88a934e67889 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:19.389405 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-850e2d90-f331-4f2c-b8e2-88a934e67889 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:19.391640 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0547170f-5c25-47a3-941b-c7b282ae9501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:19.392278 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547170f-5c25-47a3-941b-c7b282ae9501 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:19.392558 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547170f-5c25-47a3-941b-c7b282ae9501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:19.392854 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547170f-5c25-47a3-941b-c7b282ae9501 admin admin] PATH_INFO: `/v3/users/6a2da2d31f4946d3b5d94d5af886991c` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:19.395892 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0547170f-5c25-47a3-941b-c7b282ae9501 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a2da2d31f4946d3b5d94d5af886991c)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:19.397848 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0547170f-5c25-47a3-941b-c7b282ae9501 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:19.404210 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850e2d90-f331-4f2c-b8e2-88a934e67889 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:19.409295 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ffeb0-a7b3-48a0-bf45-73f607a692ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:19.409926 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ffeb0-a7b3-48a0-bf45-73f607a692ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:19.410209 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ffeb0-a7b3-48a0-bf45-73f607a692ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:19.410488 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ffeb0-a7b3-48a0-bf45-73f607a692ee admin admin] PATH_INFO: `/v3/users/d86f105235f14b3f9129f732fed4f85d` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:19.411713 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264ffeb0-a7b3-48a0-bf45-73f607a692ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=d86f105235f14b3f9129f732fed4f85d)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:19.413633 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264ffeb0-a7b3-48a0-bf45-73f607a692ee admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:19.447256 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-0547170f-5c25-47a3-941b-c7b282ae9501 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a2da2d31f4946d3b5d94d5af886991c'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:19.448033 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-0547170f-5c25-47a3-941b-c7b282ae9501 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a2da2d31f4946d3b5d94d5af886991c'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:19.462431 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-0547170f-5c25-47a3-941b-c7b282ae9501 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a2da2d31f4946d3b5d94d5af886991c'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:19.463314 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-264ffeb0-a7b3-48a0-bf45-73f607a692ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd86f105235f14b3f9129f732fed4f85d'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:19.463993 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-264ffeb0-a7b3-48a0-bf45-73f607a692ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd86f105235f14b3f9129f732fed4f85d'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:19.470137 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-0547170f-5c25-47a3-941b-c7b282ae9501 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a2da2d31f4946d3b5d94d5af886991c'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:19.472113 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-264ffeb0-a7b3-48a0-bf45-73f607a692ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd86f105235f14b3f9129f732fed4f85d'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:19.477266 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-0547170f-5c25-47a3-941b-c7b282ae9501 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a2da2d31f4946d3b5d94d5af886991c'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:19.479908 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-264ffeb0-a7b3-48a0-bf45-73f607a692ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd86f105235f14b3f9129f732fed4f85d'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:19.480554 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-264ffeb0-a7b3-48a0-bf45-73f607a692ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd86f105235f14b3f9129f732fed4f85d'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:19.483847 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1010/2035] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:43:19 2022] DELETE /identity/v3/users/6a2da2d31f4946d3b5d94d5af886991c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:19.499863 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1026/2036] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:43:19 2022] DELETE /identity/v3/users/d86f105235f14b3f9129f732fed4f85d => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:43:19.504318 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2986f6a4-965b-4523-9216-355cef05bd25 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:19.504608 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2986f6a4-965b-4523-9216-355cef05bd25 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:19.504887 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2986f6a4-965b-4523-9216-355cef05bd25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:19.526582 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-2986f6a4-965b-4523-9216-355cef05bd25 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:19.527918 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-2986f6a4-965b-4523-9216-355cef05bd25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:19.561436 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:19.601925 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1011/2037] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:19.612500 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ffeb0-a7b3-48a0-bf45-73f607a692ee admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:19.613987 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-264ffeb0-a7b3-48a0-bf45-73f607a692ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:19.675186 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ffeb0-a7b3-48a0-bf45-73f607a692ee admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:19.678960 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1529d77f-67f4-4b80-b837-829b687faa47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:19.679600 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1529d77f-67f4-4b80-b837-829b687faa47 service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:19.679882 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1529d77f-67f4-4b80-b837-829b687faa47 service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:19.680150 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1529d77f-67f4-4b80-b837-829b687faa47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:19.691052 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1529d77f-67f4-4b80-b837-829b687faa47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:19.692519 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1529d77f-67f4-4b80-b837-829b687faa47 service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:19.702438 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1027/2038] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:43:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:43:19.821971 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2986f6a4-965b-4523-9216-355cef05bd25 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:19.823444 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-2986f6a4-965b-4523-9216-355cef05bd25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:19.836092 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2986f6a4-965b-4523-9216-355cef05bd25 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:19.840952 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4313e33b-d87d-4e40-b04a-916feaa31d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:19.842129 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4313e33b-d87d-4e40-b04a-916feaa31d78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:19.842405 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4313e33b-d87d-4e40-b04a-916feaa31d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:19.842827 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4313e33b-d87d-4e40-b04a-916feaa31d78 admin admin] PATH_INFO: `/v3/projects/f1b728aa400d47fca5218f9e8b693959` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:19.843996 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4313e33b-d87d-4e40-b04a-916feaa31d78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1b728aa400d47fca5218f9e8b693959)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:19.845910 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4313e33b-d87d-4e40-b04a-916feaa31d78 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:19.902781 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-4313e33b-d87d-4e40-b04a-916feaa31d78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1b728aa400d47fca5218f9e8b693959', 'request_id': 'req-4313e33b-d87d-4e40-b04a-916feaa31d78'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:19.913737 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-4313e33b-d87d-4e40-b04a-916feaa31d78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1b728aa400d47fca5218f9e8b693959', 'request_id': 'req-4313e33b-d87d-4e40-b04a-916feaa31d78'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:19.921765 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-4313e33b-d87d-4e40-b04a-916feaa31d78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1b728aa400d47fca5218f9e8b693959', 'request_id': 'req-4313e33b-d87d-4e40-b04a-916feaa31d78'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:19.936784 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-4313e33b-d87d-4e40-b04a-916feaa31d78 admin admin] The token cache is being invalidate because project f1b728aa400d47fca5218f9e8b693959 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:43:19.937222 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-4313e33b-d87d-4e40-b04a-916feaa31d78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:19.937874 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-4313e33b-d87d-4e40-b04a-916feaa31d78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:19.939727 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1012/2039] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:43:19 2022] DELETE /identity/v3/projects/f1b728aa400d47fca5218f9e8b693959 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:19.944534 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d11be-45ed-44a3-85de-a0688be47558 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:19.944823 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d11be-45ed-44a3-85de-a0688be47558 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:19.945097 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d11be-45ed-44a3-85de-a0688be47558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:19.965709 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-932d11be-45ed-44a3-85de-a0688be47558 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:19.967478 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-932d11be-45ed-44a3-85de-a0688be47558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:20.010127 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:20.032517 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1028/2040] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:20.040215 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfeab27-a6f7-4bd7-8ced-8a6a4f1ac4fe None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:20.040503 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfeab27-a6f7-4bd7-8ced-8a6a4f1ac4fe None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:20.040779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfeab27-a6f7-4bd7-8ced-8a6a4f1ac4fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:20.061018 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-0bfeab27-a6f7-4bd7-8ced-8a6a4f1ac4fe None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:20.062226 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-0bfeab27-a6f7-4bd7-8ced-8a6a4f1ac4fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:20.070521 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1013/2041] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:20.074641 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d11be-45ed-44a3-85de-a0688be47558 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:20.084931 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d11be-45ed-44a3-85de-a0688be47558 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:20.089516 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c571623-a7fe-45cc-bae9-0ce8b10f1bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:20.090119 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c571623-a7fe-45cc-bae9-0ce8b10f1bd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:20.090397 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c571623-a7fe-45cc-bae9-0ce8b10f1bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:20.090668 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c571623-a7fe-45cc-bae9-0ce8b10f1bd0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:20.091343 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c571623-a7fe-45cc-bae9-0ce8b10f1bd0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:43:20.091643 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c571623-a7fe-45cc-bae9-0ce8b10f1bd0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:20.093227 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c571623-a7fe-45cc-bae9-0ce8b10f1bd0 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:20.108696 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1029/2042] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 14:43:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:43:20.110234 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfeab27-a6f7-4bd7-8ced-8a6a4f1ac4fe None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:20.119957 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfeab27-a6f7-4bd7-8ced-8a6a4f1ac4fe None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:20.126026 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fcf0d6-2f56-4a76-b68c-f30011b5ff8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:20.126932 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fcf0d6-2f56-4a76-b68c-f30011b5ff8d admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:20.127320 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fcf0d6-2f56-4a76-b68c-f30011b5ff8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:20.127698 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fcf0d6-2f56-4a76-b68c-f30011b5ff8d admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:20.128342 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fcf0d6-2f56-4a76-b68c-f30011b5ff8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:20.130603 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fcf0d6-2f56-4a76-b68c-f30011b5ff8d admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:20.153176 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c571623-a7fe-45cc-bae9-0ce8b10f1bd0 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:20.155992 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-5c571623-a7fe-45cc-bae9-0ce8b10f1bd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:20.165345 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1014/2043] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:43:20 2022] POST /identity/v3/projects => generated 389 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:20.170501 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fcf0d6-2f56-4a76-b68c-f30011b5ff8d admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:20.174081 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c571623-a7fe-45cc-bae9-0ce8b10f1bd0 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:20.239310 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fcf0d6-2f56-4a76-b68c-f30011b5ff8d admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:20.244421 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c3c860-3102-477a-a2b7-a58751cd2cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:20.245150 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3c860-3102-477a-a2b7-a58751cd2cc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:20.245728 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3c860-3102-477a-a2b7-a58751cd2cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:20.246107 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3c860-3102-477a-a2b7-a58751cd2cc5 admin admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:20.246974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c3c860-3102-477a-a2b7-a58751cd2cc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:20.247939 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1729d37a-7a70-45de-9ae7-efcbc0167e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:20.248883 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729d37a-7a70-45de-9ae7-efcbc0167e9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:20.249165 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729d37a-7a70-45de-9ae7-efcbc0167e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:20.249470 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729d37a-7a70-45de-9ae7-efcbc0167e9a admin admin] PATH_INFO: `/v3/projects/98d83b7b8f624a83ba12ceb9c2fbec0a` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:20.249573 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c3c860-3102-477a-a2b7-a58751cd2cc5 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:20.250628 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1729d37a-7a70-45de-9ae7-efcbc0167e9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=98d83b7b8f624a83ba12ceb9c2fbec0a)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:20.253158 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1729d37a-7a70-45de-9ae7-efcbc0167e9a admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:20.276615 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1015/2044] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:43:20 2022] POST /identity/v3/users => generated 295 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:43:20.281214 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c3c860-3102-477a-a2b7-a58751cd2cc5 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:20.292022 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c3c860-3102-477a-a2b7-a58751cd2cc5 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:20.301233 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:20.306025 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-1729d37a-7a70-45de-9ae7-efcbc0167e9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98d83b7b8f624a83ba12ceb9c2fbec0a', 'request_id': 'req-1729d37a-7a70-45de-9ae7-efcbc0167e9a'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:20.312529 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-1729d37a-7a70-45de-9ae7-efcbc0167e9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98d83b7b8f624a83ba12ceb9c2fbec0a', 'request_id': 'req-1729d37a-7a70-45de-9ae7-efcbc0167e9a'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:20.321504 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-1729d37a-7a70-45de-9ae7-efcbc0167e9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98d83b7b8f624a83ba12ceb9c2fbec0a', 'request_id': 'req-1729d37a-7a70-45de-9ae7-efcbc0167e9a'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:20.335057 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-1729d37a-7a70-45de-9ae7-efcbc0167e9a admin admin] The token cache is being invalidate because project 98d83b7b8f624a83ba12ceb9c2fbec0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:43:20.335449 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-1729d37a-7a70-45de-9ae7-efcbc0167e9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:20.336137 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-1729d37a-7a70-45de-9ae7-efcbc0167e9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:20.337733 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1030/2045] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:43:20 2022] DELETE /identity/v3/projects/98d83b7b8f624a83ba12ceb9c2fbec0a => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:43:20.342361 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1729d37a-7a70-45de-9ae7-efcbc0167e9a admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:20.344112 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-1729d37a-7a70-45de-9ae7-efcbc0167e9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:20.355604 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724b45b8-592d-47be-a2a9-febb2470546c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:20.359490 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b45b8-592d-47be-a2a9-febb2470546c admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:20.359782 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b45b8-592d-47be-a2a9-febb2470546c admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:20.360061 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b45b8-592d-47be-a2a9-febb2470546c admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:20.360726 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724b45b8-592d-47be-a2a9-febb2470546c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:20.362382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724b45b8-592d-47be-a2a9-febb2470546c admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:20.377624 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1016/2046] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:20 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:43:20.379034 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724b45b8-592d-47be-a2a9-febb2470546c admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:20.381723 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-724b45b8-592d-47be-a2a9-febb2470546c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:20.394767 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724b45b8-592d-47be-a2a9-febb2470546c admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:20.400068 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f885b7a-4dfe-44cd-9508-e65c0a67e5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:20.400769 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f885b7a-4dfe-44cd-9508-e65c0a67e5ef admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:20.401087 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f885b7a-4dfe-44cd-9508-e65c0a67e5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:20.401404 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f885b7a-4dfe-44cd-9508-e65c0a67e5ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:20.402144 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f885b7a-4dfe-44cd-9508-e65c0a67e5ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:20.404711 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f885b7a-4dfe-44cd-9508-e65c0a67e5ef admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:20.411447 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1729d37a-7a70-45de-9ae7-efcbc0167e9a admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:20.415974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1017/2047] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:20 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:43:20.416221 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2afee5c-5560-4aac-9d03-fdf7745f9863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:20.416787 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2afee5c-5560-4aac-9d03-fdf7745f9863 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:20.417061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2afee5c-5560-4aac-9d03-fdf7745f9863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:20.417327 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2afee5c-5560-4aac-9d03-fdf7745f9863 admin admin] PATH_INFO: `/v3/users/fb1f84441c2f46bfb50da9e58ddb0a7b` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:20.418455 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2afee5c-5560-4aac-9d03-fdf7745f9863 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb1f84441c2f46bfb50da9e58ddb0a7b)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:20.420337 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f885b7a-4dfe-44cd-9508-e65c0a67e5ef admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:20.420448 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2afee5c-5560-4aac-9d03-fdf7745f9863 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:20.422899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:20.430967 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f885b7a-4dfe-44cd-9508-e65c0a67e5ef admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:20.435893 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f4b263-f38d-4f53-9c6d-62a799ca865c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:20.436566 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4b263-f38d-4f53-9c6d-62a799ca865c admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:20.436850 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4b263-f38d-4f53-9c6d-62a799ca865c admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:20.437123 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4b263-f38d-4f53-9c6d-62a799ca865c admin admin] PATH_INFO: `/v3/projects/9a9e62eccf8f4bff9639137f31a36307/users/3e658197384b474bb7e02c201812ac49/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:20.455509 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f4b263-f38d-4f53-9c6d-62a799ca865c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a9e62eccf8f4bff9639137f31a36307, user_id=3e658197384b474bb7e02c201812ac49, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:20.458825 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f4b263-f38d-4f53-9c6d-62a799ca865c admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:20.472819 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-b2afee5c-5560-4aac-9d03-fdf7745f9863 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb1f84441c2f46bfb50da9e58ddb0a7b'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:20.472985 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1018/2048] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:20 2022] PUT /identity/v3/projects/9a9e62eccf8f4bff9639137f31a36307/users/3e658197384b474bb7e02c201812ac49/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:43:20.473525 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-b2afee5c-5560-4aac-9d03-fdf7745f9863 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb1f84441c2f46bfb50da9e58ddb0a7b'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:20.480805 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-b2afee5c-5560-4aac-9d03-fdf7745f9863 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb1f84441c2f46bfb50da9e58ddb0a7b'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:20.484261 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef155d8a-c514-4a2e-95c9-89f10e27bec3 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:20.484548 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef155d8a-c514-4a2e-95c9-89f10e27bec3 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:20.484827 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef155d8a-c514-4a2e-95c9-89f10e27bec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:20.488069 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-b2afee5c-5560-4aac-9d03-fdf7745f9863 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb1f84441c2f46bfb50da9e58ddb0a7b'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:20.488834 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-b2afee5c-5560-4aac-9d03-fdf7745f9863 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb1f84441c2f46bfb50da9e58ddb0a7b'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:20.507368 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1031/2049] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:43:20 2022] DELETE /identity/v3/users/fb1f84441c2f46bfb50da9e58ddb0a7b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:43:20.514212 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-ef155d8a-c514-4a2e-95c9-89f10e27bec3 None None] MFA Rules not processed for user `3e658197384b474bb7e02c201812ac49`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:20.515475 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-ef155d8a-c514-4a2e-95c9-89f10e27bec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:20.592442 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1019/2050] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:20 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:43:20.593459 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2afee5c-5560-4aac-9d03-fdf7745f9863 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:20.594914 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-b2afee5c-5560-4aac-9d03-fdf7745f9863 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:20.670772 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2afee5c-5560-4aac-9d03-fdf7745f9863 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:20.675669 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3897c9a8-5100-412a-8dec-b8cc2956a91a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:20.676286 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897c9a8-5100-412a-8dec-b8cc2956a91a admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:20.676899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897c9a8-5100-412a-8dec-b8cc2956a91a admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:20.676899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897c9a8-5100-412a-8dec-b8cc2956a91a admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:20.677475 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3897c9a8-5100-412a-8dec-b8cc2956a91a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:20.679674 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3897c9a8-5100-412a-8dec-b8cc2956a91a admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:20.710344 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1032/2051] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:43:20 2022] POST /identity/v3/projects => generated 383 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:20.714340 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef155d8a-c514-4a2e-95c9-89f10e27bec3 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:20.725778 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef155d8a-c514-4a2e-95c9-89f10e27bec3 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:20.790831 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3897c9a8-5100-412a-8dec-b8cc2956a91a admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:20.792778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-3897c9a8-5100-412a-8dec-b8cc2956a91a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:20.795709 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5cd1ba-d31c-4cb7-8320-a81afe4f49f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:20.796425 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5cd1ba-d31c-4cb7-8320-a81afe4f49f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:20.796706 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5cd1ba-d31c-4cb7-8320-a81afe4f49f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:20.796976 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5cd1ba-d31c-4cb7-8320-a81afe4f49f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:20.797611 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5cd1ba-d31c-4cb7-8320-a81afe4f49f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:20.799838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5cd1ba-d31c-4cb7-8320-a81afe4f49f8 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:20.808745 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3897c9a8-5100-412a-8dec-b8cc2956a91a admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:20.811114 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:20.814320 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b4f52f-7b2e-44d5-89b6-96082e00b58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:20.815137 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4f52f-7b2e-44d5-89b6-96082e00b58f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:20.815775 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4f52f-7b2e-44d5-89b6-96082e00b58f admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:20.816121 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4f52f-7b2e-44d5-89b6-96082e00b58f admin admin] PATH_INFO: `/v3/projects/bcbb99fcea244c11bbe902c50b3f1614` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:20.817591 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b4f52f-7b2e-44d5-89b6-96082e00b58f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcbb99fcea244c11bbe902c50b3f1614)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:20.819615 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b4f52f-7b2e-44d5-89b6-96082e00b58f admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:20.826639 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1020/2052] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:43:20 2022] POST /identity/v3/users => generated 298 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:20.830695 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5cd1ba-d31c-4cb7-8320-a81afe4f49f8 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:20.840347 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5cd1ba-d31c-4cb7-8320-a81afe4f49f8 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:20.845641 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084c531-2285-4110-9c13-40f8a8722498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:20.846236 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084c531-2285-4110-9c13-40f8a8722498 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:20.846522 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084c531-2285-4110-9c13-40f8a8722498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:20.846800 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084c531-2285-4110-9c13-40f8a8722498 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:20.847470 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e084c531-2285-4110-9c13-40f8a8722498 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:20.849060 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e084c531-2285-4110-9c13-40f8a8722498 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:20.861035 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1021/2053] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:20.865221 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084c531-2285-4110-9c13-40f8a8722498 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:20.875237 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084c531-2285-4110-9c13-40f8a8722498 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:20.876753 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-35b4f52f-7b2e-44d5-89b6-96082e00b58f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcbb99fcea244c11bbe902c50b3f1614', 'request_id': 'req-35b4f52f-7b2e-44d5-89b6-96082e00b58f'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:20.883089 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-35b4f52f-7b2e-44d5-89b6-96082e00b58f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcbb99fcea244c11bbe902c50b3f1614', 'request_id': 'req-35b4f52f-7b2e-44d5-89b6-96082e00b58f'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:20.884959 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:20.891026 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-35b4f52f-7b2e-44d5-89b6-96082e00b58f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcbb99fcea244c11bbe902c50b3f1614', 'request_id': 'req-35b4f52f-7b2e-44d5-89b6-96082e00b58f'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:20.904394 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-35b4f52f-7b2e-44d5-89b6-96082e00b58f admin admin] The token cache is being invalidate because project bcbb99fcea244c11bbe902c50b3f1614 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:43:20.904710 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-35b4f52f-7b2e-44d5-89b6-96082e00b58f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:20.905354 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-35b4f52f-7b2e-44d5-89b6-96082e00b58f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:20.906868 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1033/2054] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:43:20 2022] DELETE /identity/v3/projects/bcbb99fcea244c11bbe902c50b3f1614 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:43:20.912513 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fdf13e-3781-4e3a-881b-3ae134dff833 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:20.912815 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fdf13e-3781-4e3a-881b-3ae134dff833 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:20.913084 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fdf13e-3781-4e3a-881b-3ae134dff833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:20.933967 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-09fdf13e-3781-4e3a-881b-3ae134dff833 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:20.934179 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aa8028-6e12-4c47-bf9e-d7f9f1c804a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:20.934840 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa8028-6e12-4c47-bf9e-d7f9f1c804a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:20.935297 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-09fdf13e-3781-4e3a-881b-3ae134dff833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:20.935413 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa8028-6e12-4c47-bf9e-d7f9f1c804a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:20.935740 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa8028-6e12-4c47-bf9e-d7f9f1c804a4 admin admin] PATH_INFO: `/v3/projects/8436035c21eb43728b24c5f10ca1236f/users/6e4f5aafdd8e45e3a2d395f0381e8c04/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:20.943552 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1034/2055] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:43:20.950455 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72aa8028-6e12-4c47-bf9e-d7f9f1c804a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8436035c21eb43728b24c5f10ca1236f, user_id=6e4f5aafdd8e45e3a2d395f0381e8c04, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:20.952099 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db057f3c-cacb-41d5-a162-4c03988230d5 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:20.952391 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db057f3c-cacb-41d5-a162-4c03988230d5 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:20.952668 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db057f3c-cacb-41d5-a162-4c03988230d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:20.953950 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72aa8028-6e12-4c47-bf9e-d7f9f1c804a4 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:20.968641 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1022/2056] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:20 2022] PUT /identity/v3/projects/8436035c21eb43728b24c5f10ca1236f/users/6e4f5aafdd8e45e3a2d395f0381e8c04/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:43:20.972636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aa8028-6e12-4c47-bf9e-d7f9f1c804a4 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:20.973391 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-db057f3c-cacb-41d5-a162-4c03988230d5 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:20.974329 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-72aa8028-6e12-4c47-bf9e-d7f9f1c804a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:20.974577 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-db057f3c-cacb-41d5-a162-4c03988230d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:21.036734 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1035/2057] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:43:21.038643 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aa8028-6e12-4c47-bf9e-d7f9f1c804a4 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:21.041062 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db057f3c-cacb-41d5-a162-4c03988230d5 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:21.043473 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be82111-6fda-4ca2-b928-74e6832e75b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:21.044084 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be82111-6fda-4ca2-b928-74e6832e75b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:21.044891 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be82111-6fda-4ca2-b928-74e6832e75b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:21.045206 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be82111-6fda-4ca2-b928-74e6832e75b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:21.045825 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be82111-6fda-4ca2-b928-74e6832e75b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:21.047445 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be82111-6fda-4ca2-b928-74e6832e75b6 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:21.050612 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db057f3c-cacb-41d5-a162-4c03988230d5 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:21.055418 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3c2e79-0025-47eb-88a3-c3cb5d22f28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:21.056014 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c2e79-0025-47eb-88a3-c3cb5d22f28d admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:21.056301 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c2e79-0025-47eb-88a3-c3cb5d22f28d admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:21.056569 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c2e79-0025-47eb-88a3-c3cb5d22f28d admin admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:21.057213 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3c2e79-0025-47eb-88a3-c3cb5d22f28d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:43:21.057675 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3c2e79-0025-47eb-88a3-c3cb5d22f28d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:21.057779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1023/2058] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:20 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:21.059308 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3c2e79-0025-47eb-88a3-c3cb5d22f28d admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:21.062476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be82111-6fda-4ca2-b928-74e6832e75b6 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:21.075289 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1036/2059] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 14:43:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:43:21.075446 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be82111-6fda-4ca2-b928-74e6832e75b6 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:21.076877 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3c2e79-0025-47eb-88a3-c3cb5d22f28d admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:21.077376 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d1ca1-9f13-4529-942d-b99bcb4505b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:21.078041 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d1ca1-9f13-4529-942d-b99bcb4505b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:21.078377 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d1ca1-9f13-4529-942d-b99bcb4505b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:21.078598 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d1ca1-9f13-4529-942d-b99bcb4505b4 admin admin] PATH_INFO: `/v3/projects/8436035c21eb43728b24c5f10ca1236f/users/6e4f5aafdd8e45e3a2d395f0381e8c04/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:21.080563 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700d1ca1-9f13-4529-942d-b99bcb4505b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8436035c21eb43728b24c5f10ca1236f, user_id=6e4f5aafdd8e45e3a2d395f0381e8c04, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:21.083291 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700d1ca1-9f13-4529-942d-b99bcb4505b4 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:21.087035 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3c2e79-0025-47eb-88a3-c3cb5d22f28d admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:21.092916 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63769d44-a2d7-4003-ac5b-05f5501a1858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:21.093505 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63769d44-a2d7-4003-ac5b-05f5501a1858 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:21.093784 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63769d44-a2d7-4003-ac5b-05f5501a1858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:21.094057 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63769d44-a2d7-4003-ac5b-05f5501a1858 admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:21.094147 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1024/2060] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:21 2022] PUT /identity/v3/projects/8436035c21eb43728b24c5f10ca1236f/users/6e4f5aafdd8e45e3a2d395f0381e8c04/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:43:21.099080 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63769d44-a2d7-4003-ac5b-05f5501a1858 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:21.099080 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63769d44-a2d7-4003-ac5b-05f5501a1858 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:21.104997 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae28595d-21d7-46e1-8f17-332e3adb8e0c None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:21.105328 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae28595d-21d7-46e1-8f17-332e3adb8e0c None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:21.105562 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae28595d-21d7-46e1-8f17-332e3adb8e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:21.126338 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1037/2061] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:43:21 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:43:21.130424 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63769d44-a2d7-4003-ac5b-05f5501a1858 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:21.136717 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-ae28595d-21d7-46e1-8f17-332e3adb8e0c None None] MFA Rules not processed for user `6e4f5aafdd8e45e3a2d395f0381e8c04`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:21.137939 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-ae28595d-21d7-46e1-8f17-332e3adb8e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:21.188620 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:21.203523 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63769d44-a2d7-4003-ac5b-05f5501a1858 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:21.216552 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ea7de-3384-4d94-9a70-80874aee9579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:21.217542 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ea7de-3384-4d94-9a70-80874aee9579 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:21.218257 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ea7de-3384-4d94-9a70-80874aee9579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:21.218692 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ea7de-3384-4d94-9a70-80874aee9579 admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:21.219877 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335ea7de-3384-4d94-9a70-80874aee9579 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:21.223639 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335ea7de-3384-4d94-9a70-80874aee9579 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:21.243247 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1025/2062] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:21 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:43:21.256864 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae28595d-21d7-46e1-8f17-332e3adb8e0c None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:21.257923 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1038/2063] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:43:21 2022] POST /identity/v3/users => generated 290 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:43:21.258322 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-ae28595d-21d7-46e1-8f17-332e3adb8e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:21.261708 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ea7de-3384-4d94-9a70-80874aee9579 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:21.273513 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ea7de-3384-4d94-9a70-80874aee9579 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:21.281926 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927c63e-2e49-4979-bff8-79fa5843e3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:21.282755 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927c63e-2e49-4979-bff8-79fa5843e3c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:21.283241 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927c63e-2e49-4979-bff8-79fa5843e3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:21.283539 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927c63e-2e49-4979-bff8-79fa5843e3c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:21.284201 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4927c63e-2e49-4979-bff8-79fa5843e3c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:21.285860 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4927c63e-2e49-4979-bff8-79fa5843e3c2 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:21.298938 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1039/2064] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:43:21.302703 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927c63e-2e49-4979-bff8-79fa5843e3c2 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:21.312874 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927c63e-2e49-4979-bff8-79fa5843e3c2 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:21.318499 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3239a745-0826-4fdd-bc05-ae0d2fc51750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:21.318660 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3239a745-0826-4fdd-bc05-ae0d2fc51750 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:21.318935 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3239a745-0826-4fdd-bc05-ae0d2fc51750 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:21.319241 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3239a745-0826-4fdd-bc05-ae0d2fc51750 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:21.319889 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3239a745-0826-4fdd-bc05-ae0d2fc51750 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:21.321482 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3239a745-0826-4fdd-bc05-ae0d2fc51750 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:21.322731 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae28595d-21d7-46e1-8f17-332e3adb8e0c None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:21.329445 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed267ad-8d32-45b2-b03f-c2b7964a941f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:21.329982 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed267ad-8d32-45b2-b03f-c2b7964a941f service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:21.330264 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed267ad-8d32-45b2-b03f-c2b7964a941f service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:21.330533 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed267ad-8d32-45b2-b03f-c2b7964a941f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:21.333492 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1040/2065] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:43:21.337602 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3239a745-0826-4fdd-bc05-ae0d2fc51750 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:21.340735 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed267ad-8d32-45b2-b03f-c2b7964a941f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:21.342697 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed267ad-8d32-45b2-b03f-c2b7964a941f service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:21.348369 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3239a745-0826-4fdd-bc05-ae0d2fc51750 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:21.352537 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1026/2066] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:43:21 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:43:21.354945 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61521476-7ce9-4ebe-9c36-d8402bb2acbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:21.355630 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61521476-7ce9-4ebe-9c36-d8402bb2acbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:21.355913 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61521476-7ce9-4ebe-9c36-d8402bb2acbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:21.356190 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61521476-7ce9-4ebe-9c36-d8402bb2acbe admin admin] PATH_INFO: `/v3/projects/24fb6ed68f4644c284eee691a5ef71e0/users/354d13be721a42e5804f03ac34b13f38/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:21.370212 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:21.373437 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61521476-7ce9-4ebe-9c36-d8402bb2acbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=24fb6ed68f4644c284eee691a5ef71e0, user_id=354d13be721a42e5804f03ac34b13f38, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:21.376118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61521476-7ce9-4ebe-9c36-d8402bb2acbe admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:21.390983 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1041/2067] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:21 2022] PUT /identity/v3/projects/24fb6ed68f4644c284eee691a5ef71e0/users/354d13be721a42e5804f03ac34b13f38/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:43:21.401565 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd676f-9ccb-4be5-8f2c-335bcb09502e None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:21.402271 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd676f-9ccb-4be5-8f2c-335bcb09502e None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:21.402271 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd676f-9ccb-4be5-8f2c-335bcb09502e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:21.443551 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-21cd676f-9ccb-4be5-8f2c-335bcb09502e None None] MFA Rules not processed for user `354d13be721a42e5804f03ac34b13f38`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:21.444777 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-21cd676f-9ccb-4be5-8f2c-335bcb09502e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:21.516420 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1027/2068] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:21.524335 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61521476-7ce9-4ebe-9c36-d8402bb2acbe admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:21.525345 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cd676f-9ccb-4be5-8f2c-335bcb09502e None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:21.583860 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61521476-7ce9-4ebe-9c36-d8402bb2acbe admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:21.588664 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3064b20-c909-4a76-b2d8-ed793ad3344e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:21.589242 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cd676f-9ccb-4be5-8f2c-335bcb09502e None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:21.589371 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3064b20-c909-4a76-b2d8-ed793ad3344e admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:21.589534 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3064b20-c909-4a76-b2d8-ed793ad3344e admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:21.589868 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3064b20-c909-4a76-b2d8-ed793ad3344e admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:21.590438 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3064b20-c909-4a76-b2d8-ed793ad3344e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:21.593198 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3064b20-c909-4a76-b2d8-ed793ad3344e admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:21.594473 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b633e847-c2ac-4d4d-b689-cdcf28922a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:21.595102 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633e847-c2ac-4d4d-b689-cdcf28922a53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:21.595424 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633e847-c2ac-4d4d-b689-cdcf28922a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:21.595695 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633e847-c2ac-4d4d-b689-cdcf28922a53 admin admin] PATH_INFO: `/v3/users/3e658197384b474bb7e02c201812ac49` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:21.596835 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b633e847-c2ac-4d4d-b689-cdcf28922a53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e658197384b474bb7e02c201812ac49)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:21.598823 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b633e847-c2ac-4d4d-b689-cdcf28922a53 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:21.605678 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:21.621748 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1042/2069] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:43:21 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:21.626288 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3064b20-c909-4a76-b2d8-ed793ad3344e admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:21.653231 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b633e847-c2ac-4d4d-b689-cdcf28922a53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e658197384b474bb7e02c201812ac49'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:21.654443 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b633e847-c2ac-4d4d-b689-cdcf28922a53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e658197384b474bb7e02c201812ac49'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:21.670424 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b633e847-c2ac-4d4d-b689-cdcf28922a53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e658197384b474bb7e02c201812ac49'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:21.679747 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b633e847-c2ac-4d4d-b689-cdcf28922a53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e658197384b474bb7e02c201812ac49'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:21.691434 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b633e847-c2ac-4d4d-b689-cdcf28922a53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e658197384b474bb7e02c201812ac49'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:21.698394 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1028/2070] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:43:21 2022] DELETE /identity/v3/users/3e658197384b474bb7e02c201812ac49 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:21.699609 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3064b20-c909-4a76-b2d8-ed793ad3344e admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:21.703403 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d55b483-7008-4e20-bf9d-c483b94cd5cc None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:21.703691 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d55b483-7008-4e20-bf9d-c483b94cd5cc None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:21.703965 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d55b483-7008-4e20-bf9d-c483b94cd5cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:21.723797 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-2d55b483-7008-4e20-bf9d-c483b94cd5cc None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:21.725007 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-2d55b483-7008-4e20-bf9d-c483b94cd5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:21.759842 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d993102-799b-488d-b9a6-97156a3a90da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:21.760757 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d993102-799b-488d-b9a6-97156a3a90da admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:21.761036 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d993102-799b-488d-b9a6-97156a3a90da admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:21.761302 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d993102-799b-488d-b9a6-97156a3a90da admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:21.761931 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d993102-799b-488d-b9a6-97156a3a90da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:21.765453 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d993102-799b-488d-b9a6-97156a3a90da admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:21.791906 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1029/2071] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:43:21.793312 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1043/2072] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:43:21 2022] POST /identity/v3/users => generated 296 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:21.796253 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d55b483-7008-4e20-bf9d-c483b94cd5cc None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:21.797749 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-2d55b483-7008-4e20-bf9d-c483b94cd5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:21.798950 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d993102-799b-488d-b9a6-97156a3a90da admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:21.800679 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-1d993102-799b-488d-b9a6-97156a3a90da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:21.810090 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d55b483-7008-4e20-bf9d-c483b94cd5cc None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:21.814930 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f2e95-2a7a-46a5-b2b3-699433421bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:21.815747 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:21.815877 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f2e95-2a7a-46a5-b2b3-699433421bc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:21.816348 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f2e95-2a7a-46a5-b2b3-699433421bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:21.816626 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f2e95-2a7a-46a5-b2b3-699433421bc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:21.817309 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f2e95-2a7a-46a5-b2b3-699433421bc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:21.819022 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f2e95-2a7a-46a5-b2b3-699433421bc7 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:21.830575 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1030/2073] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:21.834851 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f2e95-2a7a-46a5-b2b3-699433421bc7 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:21.844526 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f2e95-2a7a-46a5-b2b3-699433421bc7 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:21.849220 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26d8f5f-3a88-472a-bdd7-174ec44f727c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:21.849867 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d8f5f-3a88-472a-bdd7-174ec44f727c admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:21.850147 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d8f5f-3a88-472a-bdd7-174ec44f727c admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:21.850421 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d8f5f-3a88-472a-bdd7-174ec44f727c admin admin] PATH_INFO: `/v3/projects/dde4ff1a92214091bb54938b30b1dac0/users/6e20b84b0b1f43d4877b38d5a0bc6a60/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:21.864492 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d993102-799b-488d-b9a6-97156a3a90da admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:21.866925 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26d8f5f-3a88-472a-bdd7-174ec44f727c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dde4ff1a92214091bb54938b30b1dac0, user_id=6e20b84b0b1f43d4877b38d5a0bc6a60, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:21.868424 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcc8754-4634-40e1-866e-23446bb6b9fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:21.869040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc8754-4634-40e1-866e-23446bb6b9fa service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:21.869320 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc8754-4634-40e1-866e-23446bb6b9fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:21.869592 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc8754-4634-40e1-866e-23446bb6b9fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:21.869881 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26d8f5f-3a88-472a-bdd7-174ec44f727c admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:21.880961 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcc8754-4634-40e1-866e-23446bb6b9fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:21.883070 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcc8754-4634-40e1-866e-23446bb6b9fa service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:21.886269 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1031/2074] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:21 2022] PUT /identity/v3/projects/dde4ff1a92214091bb54938b30b1dac0/users/6e20b84b0b1f43d4877b38d5a0bc6a60/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:43:21.896312 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26d8f5f-3a88-472a-bdd7-174ec44f727c admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:21.900642 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1044/2075] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:43:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:43:21.962156 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:21.981015 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26d8f5f-3a88-472a-bdd7-174ec44f727c admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:21.987767 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2163cf-dcae-4648-b57e-0d2c1d12c7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:21.988454 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2163cf-dcae-4648-b57e-0d2c1d12c7da admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:21.988740 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2163cf-dcae-4648-b57e-0d2c1d12c7da admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:21.989109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2163cf-dcae-4648-b57e-0d2c1d12c7da admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:21.989828 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2163cf-dcae-4648-b57e-0d2c1d12c7da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:21.991586 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2163cf-dcae-4648-b57e-0d2c1d12c7da admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:22.002395 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1032/2076] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:21 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:22.007272 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcc8754-4634-40e1-866e-23446bb6b9fa service neutron] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:22.018187 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcc8754-4634-40e1-866e-23446bb6b9fa service neutron] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:22.022981 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8fdd2d-0341-454d-bc7d-703d0d70b7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:22.023672 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8fdd2d-0341-454d-bc7d-703d0d70b7af admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:22.023956 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8fdd2d-0341-454d-bc7d-703d0d70b7af admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:22.024230 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8fdd2d-0341-454d-bc7d-703d0d70b7af admin admin] PATH_INFO: `/v3/projects/dde4ff1a92214091bb54938b30b1dac0/users/6e20b84b0b1f43d4877b38d5a0bc6a60/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:22.026167 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8fdd2d-0341-454d-bc7d-703d0d70b7af admin admin] RBAC: Authorizing `identity:create_grant(project_id=dde4ff1a92214091bb54938b30b1dac0, user_id=6e20b84b0b1f43d4877b38d5a0bc6a60, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:22.029402 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8fdd2d-0341-454d-bc7d-703d0d70b7af admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:22.040482 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1045/2077] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:22 2022] PUT /identity/v3/projects/dde4ff1a92214091bb54938b30b1dac0/users/6e20b84b0b1f43d4877b38d5a0bc6a60/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:22.052978 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb8893-281f-414d-9b33-a5d818d7d5cb None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:22.053171 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb8893-281f-414d-9b33-a5d818d7d5cb None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:22.053551 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb8893-281f-414d-9b33-a5d818d7d5cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:22.084968 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-39eb8893-281f-414d-9b33-a5d818d7d5cb None None] MFA Rules not processed for user `6e20b84b0b1f43d4877b38d5a0bc6a60`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:22.086530 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-39eb8893-281f-414d-9b33-a5d818d7d5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:22.161491 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1033/2078] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:22 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:22.174683 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8fdd2d-0341-454d-bc7d-703d0d70b7af admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:22.176453 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-1d8fdd2d-0341-454d-bc7d-703d0d70b7af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:22.238226 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8fdd2d-0341-454d-bc7d-703d0d70b7af admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:22.242169 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88772448-afe5-43fd-8985-c590a197b849 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:22.242868 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88772448-afe5-43fd-8985-c590a197b849 service cinder] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:22.243057 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88772448-afe5-43fd-8985-c590a197b849 service cinder] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:22.243364 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88772448-afe5-43fd-8985-c590a197b849 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:22.254382 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88772448-afe5-43fd-8985-c590a197b849 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:22.255855 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88772448-afe5-43fd-8985-c590a197b849 service cinder] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:22.265833 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1046/2079] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:43:22 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:43:22.436734 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eb8893-281f-414d-9b33-a5d818d7d5cb None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:22.438216 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-39eb8893-281f-414d-9b33-a5d818d7d5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:22.514772 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eb8893-281f-414d-9b33-a5d818d7d5cb None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:22.521661 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a992893e-d969-41ec-87a8-4bcaf2399a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:22.522644 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992893e-d969-41ec-87a8-4bcaf2399a6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:22.522838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992893e-d969-41ec-87a8-4bcaf2399a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:22.523350 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992893e-d969-41ec-87a8-4bcaf2399a6f admin admin] PATH_INFO: `/v3/projects/9a9e62eccf8f4bff9639137f31a36307` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:22.524911 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a992893e-d969-41ec-87a8-4bcaf2399a6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a9e62eccf8f4bff9639137f31a36307)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:22.527386 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a992893e-d969-41ec-87a8-4bcaf2399a6f admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:22.605665 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-a992893e-d969-41ec-87a8-4bcaf2399a6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a9e62eccf8f4bff9639137f31a36307', 'request_id': 'req-a992893e-d969-41ec-87a8-4bcaf2399a6f'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:22.612853 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-a992893e-d969-41ec-87a8-4bcaf2399a6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a9e62eccf8f4bff9639137f31a36307', 'request_id': 'req-a992893e-d969-41ec-87a8-4bcaf2399a6f'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:22.620609 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-a992893e-d969-41ec-87a8-4bcaf2399a6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a9e62eccf8f4bff9639137f31a36307', 'request_id': 'req-a992893e-d969-41ec-87a8-4bcaf2399a6f'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:22.636952 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-a992893e-d969-41ec-87a8-4bcaf2399a6f admin admin] The token cache is being invalidate because project 9a9e62eccf8f4bff9639137f31a36307 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:43:22.637341 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-a992893e-d969-41ec-87a8-4bcaf2399a6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:22.638861 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-a992893e-d969-41ec-87a8-4bcaf2399a6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:22.640526 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1034/2080] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:43:22 2022] DELETE /identity/v3/projects/9a9e62eccf8f4bff9639137f31a36307 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:22.645134 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88772448-afe5-43fd-8985-c590a197b849 service cinder] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:22.657181 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88772448-afe5-43fd-8985-c590a197b849 service cinder] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:22.739425 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc7346-3a78-4ffe-b83f-9fdb51b0a2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:22.740115 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc7346-3a78-4ffe-b83f-9fdb51b0a2f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:22.740334 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc7346-3a78-4ffe-b83f-9fdb51b0a2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:22.741029 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc7346-3a78-4ffe-b83f-9fdb51b0a2f5 admin admin] PATH_INFO: `/v3/users/6e4f5aafdd8e45e3a2d395f0381e8c04` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:22.742179 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edc7346-3a78-4ffe-b83f-9fdb51b0a2f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e4f5aafdd8e45e3a2d395f0381e8c04)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:22.744164 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edc7346-3a78-4ffe-b83f-9fdb51b0a2f5 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:22.810872 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a992893e-d969-41ec-87a8-4bcaf2399a6f admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:22.812926 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-a992893e-d969-41ec-87a8-4bcaf2399a6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:22.819394 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:22.833260 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-2edc7346-3a78-4ffe-b83f-9fdb51b0a2f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e4f5aafdd8e45e3a2d395f0381e8c04'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:22.834376 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-2edc7346-3a78-4ffe-b83f-9fdb51b0a2f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e4f5aafdd8e45e3a2d395f0381e8c04'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:22.836382 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:22.847959 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-2edc7346-3a78-4ffe-b83f-9fdb51b0a2f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e4f5aafdd8e45e3a2d395f0381e8c04'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:22.865899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-2edc7346-3a78-4ffe-b83f-9fdb51b0a2f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e4f5aafdd8e45e3a2d395f0381e8c04'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:22.866679 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-2edc7346-3a78-4ffe-b83f-9fdb51b0a2f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e4f5aafdd8e45e3a2d395f0381e8c04'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:22.893112 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1047/2081] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:43:22 2022] DELETE /identity/v3/users/6e4f5aafdd8e45e3a2d395f0381e8c04 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:22.919513 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a992893e-d969-41ec-87a8-4bcaf2399a6f admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:23.005186 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded6129f-714d-42c3-9b65-a748480c1b2c tempest-VolumeQuotaClassesTest-650294568 tempest-VolumeQuotaClassesTest-650294568-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e20b84b0b1f43d4877b38d5a0bc6a60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dde4ff1a92214091bb54938b30b1dac0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:23.005933 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded6129f-714d-42c3-9b65-a748480c1b2c tempest-VolumeQuotaClassesTest-650294568 tempest-VolumeQuotaClassesTest-650294568-project-admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:23.006546 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded6129f-714d-42c3-9b65-a748480c1b2c tempest-VolumeQuotaClassesTest-650294568 tempest-VolumeQuotaClassesTest-650294568-project-admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:23.006546 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded6129f-714d-42c3-9b65-a748480c1b2c tempest-VolumeQuotaClassesTest-650294568 tempest-VolumeQuotaClassesTest-650294568-project-admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:23.007333 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded6129f-714d-42c3-9b65-a748480c1b2c tempest-VolumeQuotaClassesTest-650294568 tempest-VolumeQuotaClassesTest-650294568-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:43:23.007592 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded6129f-714d-42c3-9b65-a748480c1b2c tempest-VolumeQuotaClassesTest-650294568 tempest-VolumeQuotaClassesTest-650294568-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:23.009439 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded6129f-714d-42c3-9b65-a748480c1b2c tempest-VolumeQuotaClassesTest-650294568 tempest-VolumeQuotaClassesTest-650294568-project-admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:23.026076 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1035/2082] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 14:43:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:43:23.029502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc7346-3a78-4ffe-b83f-9fdb51b0a2f5 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:23.031066 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-2edc7346-3a78-4ffe-b83f-9fdb51b0a2f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:23.047142 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc7346-3a78-4ffe-b83f-9fdb51b0a2f5 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:23.053403 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8d268-7fa1-4cc0-ad2f-4172a9fa0f83 tempest-VolumeQuotaClassesTest-650294568 tempest-VolumeQuotaClassesTest-650294568-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e20b84b0b1f43d4877b38d5a0bc6a60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dde4ff1a92214091bb54938b30b1dac0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:23.053953 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8d268-7fa1-4cc0-ad2f-4172a9fa0f83 tempest-VolumeQuotaClassesTest-650294568 tempest-VolumeQuotaClassesTest-650294568-project-admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:23.055382 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8d268-7fa1-4cc0-ad2f-4172a9fa0f83 tempest-VolumeQuotaClassesTest-650294568 tempest-VolumeQuotaClassesTest-650294568-project-admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:23.055675 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8d268-7fa1-4cc0-ad2f-4172a9fa0f83 tempest-VolumeQuotaClassesTest-650294568 tempest-VolumeQuotaClassesTest-650294568-project-admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:23.056316 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e8d268-7fa1-4cc0-ad2f-4172a9fa0f83 tempest-VolumeQuotaClassesTest-650294568 tempest-VolumeQuotaClassesTest-650294568-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:23.058627 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e8d268-7fa1-4cc0-ad2f-4172a9fa0f83 tempest-VolumeQuotaClassesTest-650294568 tempest-VolumeQuotaClassesTest-650294568-project-admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:23.103319 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1048/2083] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:43:23 2022] POST /identity/v3/projects => generated 355 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:43:23.105906 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded6129f-714d-42c3-9b65-a748480c1b2c tempest-VolumeQuotaClassesTest-650294568 tempest-VolumeQuotaClassesTest-650294568-project-admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:23.117129 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded6129f-714d-42c3-9b65-a748480c1b2c tempest-VolumeQuotaClassesTest-650294568 tempest-VolumeQuotaClassesTest-650294568-project-admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:23.177733 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6bd3de-f982-4e9c-96c2-d5a915b2812a tempest-VolumeQuotaClassesTest-650294568 tempest-VolumeQuotaClassesTest-650294568-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e20b84b0b1f43d4877b38d5a0bc6a60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dde4ff1a92214091bb54938b30b1dac0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:23.178437 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6bd3de-f982-4e9c-96c2-d5a915b2812a tempest-VolumeQuotaClassesTest-650294568 tempest-VolumeQuotaClassesTest-650294568-project-admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:23.178723 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6bd3de-f982-4e9c-96c2-d5a915b2812a tempest-VolumeQuotaClassesTest-650294568 tempest-VolumeQuotaClassesTest-650294568-project-admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:23.178997 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6bd3de-f982-4e9c-96c2-d5a915b2812a tempest-VolumeQuotaClassesTest-650294568 tempest-VolumeQuotaClassesTest-650294568-project-admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:23.179692 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6bd3de-f982-4e9c-96c2-d5a915b2812a tempest-VolumeQuotaClassesTest-650294568 tempest-VolumeQuotaClassesTest-650294568-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:43:23.179996 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6bd3de-f982-4e9c-96c2-d5a915b2812a tempest-VolumeQuotaClassesTest-650294568 tempest-VolumeQuotaClassesTest-650294568-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:23.181593 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6bd3de-f982-4e9c-96c2-d5a915b2812a tempest-VolumeQuotaClassesTest-650294568 tempest-VolumeQuotaClassesTest-650294568-project-admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:23.196368 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1036/2084] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 14:43:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:43:23.212468 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8d268-7fa1-4cc0-ad2f-4172a9fa0f83 tempest-VolumeQuotaClassesTest-650294568 tempest-VolumeQuotaClassesTest-650294568-project-admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:23.214443 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-c6e8d268-7fa1-4cc0-ad2f-4172a9fa0f83 tempest-VolumeQuotaClassesTest-650294568 tempest-VolumeQuotaClassesTest-650294568-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:23.272884 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6bd3de-f982-4e9c-96c2-d5a915b2812a tempest-VolumeQuotaClassesTest-650294568 tempest-VolumeQuotaClassesTest-650294568-project-admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:23.284010 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6bd3de-f982-4e9c-96c2-d5a915b2812a tempest-VolumeQuotaClassesTest-650294568 tempest-VolumeQuotaClassesTest-650294568-project-admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:23.288768 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b45b54d-ae4e-4737-8826-16b5b2b27e97 tempest-VolumeQuotaClassesTest-650294568 tempest-VolumeQuotaClassesTest-650294568-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e20b84b0b1f43d4877b38d5a0bc6a60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dde4ff1a92214091bb54938b30b1dac0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:23.289398 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b45b54d-ae4e-4737-8826-16b5b2b27e97 tempest-VolumeQuotaClassesTest-650294568 tempest-VolumeQuotaClassesTest-650294568-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:23.289676 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b45b54d-ae4e-4737-8826-16b5b2b27e97 tempest-VolumeQuotaClassesTest-650294568 tempest-VolumeQuotaClassesTest-650294568-project-admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:23.289951 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b45b54d-ae4e-4737-8826-16b5b2b27e97 tempest-VolumeQuotaClassesTest-650294568 tempest-VolumeQuotaClassesTest-650294568-project-admin] PATH_INFO: `/v3/projects/a6fe2d6522a848fbb685276dbd9a013c` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:23.291080 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b45b54d-ae4e-4737-8826-16b5b2b27e97 tempest-VolumeQuotaClassesTest-650294568 tempest-VolumeQuotaClassesTest-650294568-project-admin] RBAC: Authorizing `identity:delete_project(project_id=a6fe2d6522a848fbb685276dbd9a013c)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:23.293019 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b45b54d-ae4e-4737-8826-16b5b2b27e97 tempest-VolumeQuotaClassesTest-650294568 tempest-VolumeQuotaClassesTest-650294568-project-admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:23.303364 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8d268-7fa1-4cc0-ad2f-4172a9fa0f83 tempest-VolumeQuotaClassesTest-650294568 tempest-VolumeQuotaClassesTest-650294568-project-admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:23.309641 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e856e49e-64b9-4b4b-b254-6654ae41869a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:23.310875 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856e49e-64b9-4b4b-b254-6654ae41869a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:23.311266 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856e49e-64b9-4b4b-b254-6654ae41869a admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:23.311756 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856e49e-64b9-4b4b-b254-6654ae41869a admin admin] PATH_INFO: `/v3/projects/8436035c21eb43728b24c5f10ca1236f` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:23.313307 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e856e49e-64b9-4b4b-b254-6654ae41869a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8436035c21eb43728b24c5f10ca1236f)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:23.317158 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e856e49e-64b9-4b4b-b254-6654ae41869a admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:23.333699 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:23.347561 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-3b45b54d-ae4e-4737-8826-16b5b2b27e97 tempest-VolumeQuotaClassesTest-650294568 tempest-VolumeQuotaClassesTest-650294568-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6fe2d6522a848fbb685276dbd9a013c', 'request_id': 'req-3b45b54d-ae4e-4737-8826-16b5b2b27e97'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:23.353485 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-3b45b54d-ae4e-4737-8826-16b5b2b27e97 tempest-VolumeQuotaClassesTest-650294568 tempest-VolumeQuotaClassesTest-650294568-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6fe2d6522a848fbb685276dbd9a013c', 'request_id': 'req-3b45b54d-ae4e-4737-8826-16b5b2b27e97'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:23.362293 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-3b45b54d-ae4e-4737-8826-16b5b2b27e97 tempest-VolumeQuotaClassesTest-650294568 tempest-VolumeQuotaClassesTest-650294568-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6fe2d6522a848fbb685276dbd9a013c', 'request_id': 'req-3b45b54d-ae4e-4737-8826-16b5b2b27e97'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:23.376514 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-3b45b54d-ae4e-4737-8826-16b5b2b27e97 tempest-VolumeQuotaClassesTest-650294568 tempest-VolumeQuotaClassesTest-650294568-project-admin] The token cache is being invalidate because project a6fe2d6522a848fbb685276dbd9a013c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:43:23.376805 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-3b45b54d-ae4e-4737-8826-16b5b2b27e97 tempest-VolumeQuotaClassesTest-650294568 tempest-VolumeQuotaClassesTest-650294568-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:23.377439 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-3b45b54d-ae4e-4737-8826-16b5b2b27e97 tempest-VolumeQuotaClassesTest-650294568 tempest-VolumeQuotaClassesTest-650294568-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:23.378978 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-e856e49e-64b9-4b4b-b254-6654ae41869a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8436035c21eb43728b24c5f10ca1236f', 'request_id': 'req-e856e49e-64b9-4b4b-b254-6654ae41869a'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:23.379127 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1037/2085] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:43:23 2022] DELETE /identity/v3/projects/a6fe2d6522a848fbb685276dbd9a013c => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:43:23.386161 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-e856e49e-64b9-4b4b-b254-6654ae41869a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8436035c21eb43728b24c5f10ca1236f', 'request_id': 'req-e856e49e-64b9-4b4b-b254-6654ae41869a'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:23.395363 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-e856e49e-64b9-4b4b-b254-6654ae41869a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8436035c21eb43728b24c5f10ca1236f', 'request_id': 'req-e856e49e-64b9-4b4b-b254-6654ae41869a'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:23.412576 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-e856e49e-64b9-4b4b-b254-6654ae41869a admin admin] The token cache is being invalidate because project 8436035c21eb43728b24c5f10ca1236f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:43:23.412877 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-e856e49e-64b9-4b4b-b254-6654ae41869a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:23.413540 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-e856e49e-64b9-4b4b-b254-6654ae41869a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:23.415153 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1049/2086] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:43:23 2022] DELETE /identity/v3/projects/8436035c21eb43728b24c5f10ca1236f => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:23.420671 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23315dda-d2f3-4ce9-8ee6-24149a4bb91a None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:23.420957 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23315dda-d2f3-4ce9-8ee6-24149a4bb91a None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:23.421232 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23315dda-d2f3-4ce9-8ee6-24149a4bb91a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:23.443075 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-23315dda-d2f3-4ce9-8ee6-24149a4bb91a None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:23.444436 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-23315dda-d2f3-4ce9-8ee6-24149a4bb91a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:23.522673 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1038/2087] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:23.531173 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e9109-c830-4bc9-94bd-7c5877e12024 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:23.531547 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e9109-c830-4bc9-94bd-7c5877e12024 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:23.532015 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e9109-c830-4bc9-94bd-7c5877e12024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:23.556897 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-563e9109-c830-4bc9-94bd-7c5877e12024 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:23.558119 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-563e9109-c830-4bc9-94bd-7c5877e12024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:23.568856 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1050/2088] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:43:23.574441 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23315dda-d2f3-4ce9-8ee6-24149a4bb91a None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:23.586136 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23315dda-d2f3-4ce9-8ee6-24149a4bb91a None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:23.602344 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95caf03b-ab08-4914-906b-398a4ffd5227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:23.603127 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95caf03b-ab08-4914-906b-398a4ffd5227 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:23.603602 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95caf03b-ab08-4914-906b-398a4ffd5227 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:23.603989 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95caf03b-ab08-4914-906b-398a4ffd5227 admin admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:23.605776 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95caf03b-ab08-4914-906b-398a4ffd5227 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:43:23.606188 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95caf03b-ab08-4914-906b-398a4ffd5227 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:23.609611 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95caf03b-ab08-4914-906b-398a4ffd5227 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:23.631439 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1039/2089] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 14:43:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:43:23.637590 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e9109-c830-4bc9-94bd-7c5877e12024 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:23.648025 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e9109-c830-4bc9-94bd-7c5877e12024 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:23.661358 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19722e35-7f12-4767-a2ba-8ac2fe9d32c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:23.662204 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19722e35-7f12-4767-a2ba-8ac2fe9d32c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:23.663923 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19722e35-7f12-4767-a2ba-8ac2fe9d32c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:23.664476 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19722e35-7f12-4767-a2ba-8ac2fe9d32c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:23.665441 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19722e35-7f12-4767-a2ba-8ac2fe9d32c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:23.667803 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19722e35-7f12-4767-a2ba-8ac2fe9d32c5 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:23.668462 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95caf03b-ab08-4914-906b-398a4ffd5227 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:23.670607 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-95caf03b-ab08-4914-906b-398a4ffd5227 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:23.692592 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95caf03b-ab08-4914-906b-398a4ffd5227 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:23.698234 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ac72e-d2d3-44ff-a59a-cdd1aad8f3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:23.699086 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ac72e-d2d3-44ff-a59a-cdd1aad8f3a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:23.699575 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ac72e-d2d3-44ff-a59a-cdd1aad8f3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:23.699858 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ac72e-d2d3-44ff-a59a-cdd1aad8f3a1 admin admin] PATH_INFO: `/v3/users/354d13be721a42e5804f03ac34b13f38` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:23.701956 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ac72e-d2d3-44ff-a59a-cdd1aad8f3a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=354d13be721a42e5804f03ac34b13f38)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:23.704726 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ac72e-d2d3-44ff-a59a-cdd1aad8f3a1 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:23.716954 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1051/2090] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:43:23 2022] POST /identity/v3/projects => generated 367 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:43:23.721372 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19722e35-7f12-4767-a2ba-8ac2fe9d32c5 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:23.728778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:INSERT=4,keystone:DELETE=10,keystone:SELECT=86 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:23.781488 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-387ac72e-d2d3-44ff-a59a-cdd1aad8f3a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '354d13be721a42e5804f03ac34b13f38'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:23.782654 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-387ac72e-d2d3-44ff-a59a-cdd1aad8f3a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '354d13be721a42e5804f03ac34b13f38'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:23.807699 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-387ac72e-d2d3-44ff-a59a-cdd1aad8f3a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '354d13be721a42e5804f03ac34b13f38'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:23.827498 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-387ac72e-d2d3-44ff-a59a-cdd1aad8f3a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '354d13be721a42e5804f03ac34b13f38'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:23.838869 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19722e35-7f12-4767-a2ba-8ac2fe9d32c5 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:23.839485 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-387ac72e-d2d3-44ff-a59a-cdd1aad8f3a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '354d13be721a42e5804f03ac34b13f38'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:23.859647 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04581e5d-8b54-4e7f-a4cd-8f6bf76d4329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:23.859647 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04581e5d-8b54-4e7f-a4cd-8f6bf76d4329 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:23.859647 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04581e5d-8b54-4e7f-a4cd-8f6bf76d4329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:23.859647 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04581e5d-8b54-4e7f-a4cd-8f6bf76d4329 admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:23.859968 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04581e5d-8b54-4e7f-a4cd-8f6bf76d4329 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:23.859968 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04581e5d-8b54-4e7f-a4cd-8f6bf76d4329 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:23.876504 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1040/2091] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:43:23 2022] DELETE /identity/v3/users/354d13be721a42e5804f03ac34b13f38 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:43:23.878491 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88233aaa-107c-494d-a553-97c367700c58 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:23.881767 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88233aaa-107c-494d-a553-97c367700c58 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:23.881890 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1052/2092] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:43:23 2022] POST /identity/v3/users => generated 284 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:23.881972 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88233aaa-107c-494d-a553-97c367700c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:23.884357 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04581e5d-8b54-4e7f-a4cd-8f6bf76d4329 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:23.886565 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-04581e5d-8b54-4e7f-a4cd-8f6bf76d4329 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:23.901177 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-88233aaa-107c-494d-a553-97c367700c58 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:23.902402 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-88233aaa-107c-494d-a553-97c367700c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:23.909132 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04581e5d-8b54-4e7f-a4cd-8f6bf76d4329 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:23.924812 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:23.986058 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1041/2093] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:23.994525 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eed181e-6aa2-4761-8512-b49f30fb23b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:23.995295 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed181e-6aa2-4761-8512-b49f30fb23b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:23.995584 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed181e-6aa2-4761-8512-b49f30fb23b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:23.995861 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed181e-6aa2-4761-8512-b49f30fb23b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:23.996409 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88233aaa-107c-494d-a553-97c367700c58 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:23.996718 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eed181e-6aa2-4761-8512-b49f30fb23b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:24.009774 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-88233aaa-107c-494d-a553-97c367700c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:24.009947 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eed181e-6aa2-4761-8512-b49f30fb23b4 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:24.012618 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1053/2094] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:23 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:24.017080 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eed181e-6aa2-4761-8512-b49f30fb23b4 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:24.027811 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eed181e-6aa2-4761-8512-b49f30fb23b4 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:24.033144 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b40d5c-022a-4f23-b68c-45769857b25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:24.034137 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b40d5c-022a-4f23-b68c-45769857b25b admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:24.034137 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b40d5c-022a-4f23-b68c-45769857b25b admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:24.034335 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b40d5c-022a-4f23-b68c-45769857b25b admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:24.034922 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b40d5c-022a-4f23-b68c-45769857b25b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:24.036569 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b40d5c-022a-4f23-b68c-45769857b25b admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:24.048533 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1054/2095] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:24.052799 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b40d5c-022a-4f23-b68c-45769857b25b admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:24.061596 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88233aaa-107c-494d-a553-97c367700c58 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:24.063264 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b40d5c-022a-4f23-b68c-45769857b25b admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:24.065599 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7fe2ad-59a1-4fc5-ba55-1f1bcc613a43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:24.066201 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7fe2ad-59a1-4fc5-ba55-1f1bcc613a43 service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:24.066475 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7fe2ad-59a1-4fc5-ba55-1f1bcc613a43 service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:24.066745 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7fe2ad-59a1-4fc5-ba55-1f1bcc613a43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:24.068240 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b2bf93-495e-435f-a364-7eea14cde404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:24.068879 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2bf93-495e-435f-a364-7eea14cde404 admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:24.069214 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2bf93-495e-435f-a364-7eea14cde404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:24.069493 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2bf93-495e-435f-a364-7eea14cde404 admin admin] PATH_INFO: `/v3/projects/46715c44599c4522a88f8ca617447c9c/users/bb134500cecb4fbab05a05a279a46a9b/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:24.076277 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7fe2ad-59a1-4fc5-ba55-1f1bcc613a43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:24.077708 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7fe2ad-59a1-4fc5-ba55-1f1bcc613a43 service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:24.090173 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b2bf93-495e-435f-a364-7eea14cde404 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46715c44599c4522a88f8ca617447c9c, user_id=bb134500cecb4fbab05a05a279a46a9b, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:24.093550 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b2bf93-495e-435f-a364-7eea14cde404 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:24.094072 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1042/2096] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:43:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:43:24.114477 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1055/2097] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:24 2022] PUT /identity/v3/projects/46715c44599c4522a88f8ca617447c9c/users/bb134500cecb4fbab05a05a279a46a9b/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:43:24.129083 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21eb9de-17fa-4808-98e0-1fe733f8f22c None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:24.129463 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21eb9de-17fa-4808-98e0-1fe733f8f22c None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:24.129809 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21eb9de-17fa-4808-98e0-1fe733f8f22c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:24.162062 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-c21eb9de-17fa-4808-98e0-1fe733f8f22c None None] MFA Rules not processed for user `bb134500cecb4fbab05a05a279a46a9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:24.163464 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-c21eb9de-17fa-4808-98e0-1fe733f8f22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:24.269659 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1043/2098] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:24 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:24.274194 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b2bf93-495e-435f-a364-7eea14cde404 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:24.376838 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:24.386866 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b2bf93-495e-435f-a364-7eea14cde404 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:24.397465 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432af850-4e33-4978-bd49-9f7e2cfa7a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:24.398864 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432af850-4e33-4978-bd49-9f7e2cfa7a3e admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:24.398864 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432af850-4e33-4978-bd49-9f7e2cfa7a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:24.398864 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432af850-4e33-4978-bd49-9f7e2cfa7a3e admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:24.399704 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432af850-4e33-4978-bd49-9f7e2cfa7a3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:24.402515 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432af850-4e33-4978-bd49-9f7e2cfa7a3e admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:24.453298 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1056/2099] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:43:24 2022] POST /identity/v3/projects => generated 365 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:43:24.457682 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21eb9de-17fa-4808-98e0-1fe733f8f22c None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:24.472563 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21eb9de-17fa-4808-98e0-1fe733f8f22c None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:24.492411 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432af850-4e33-4978-bd49-9f7e2cfa7a3e admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:24.494240 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-432af850-4e33-4978-bd49-9f7e2cfa7a3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:24.567589 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:24.583671 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65119f20-2885-484d-98e6-f0d4e167783c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:24.584336 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65119f20-2885-484d-98e6-f0d4e167783c admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:24.584614 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65119f20-2885-484d-98e6-f0d4e167783c admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:24.584913 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65119f20-2885-484d-98e6-f0d4e167783c admin admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:24.585548 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65119f20-2885-484d-98e6-f0d4e167783c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:24.593911 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65119f20-2885-484d-98e6-f0d4e167783c admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:24.628049 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432af850-4e33-4978-bd49-9f7e2cfa7a3e admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:24.632973 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1044/2100] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:43:24 2022] POST /identity/v3/users => generated 289 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:43:24.636123 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8079e74d-f0b0-4bb9-887e-5a716244e047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:24.636123 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079e74d-f0b0-4bb9-887e-5a716244e047 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:24.636123 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079e74d-f0b0-4bb9-887e-5a716244e047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:24.636123 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079e74d-f0b0-4bb9-887e-5a716244e047 admin admin] PATH_INFO: `/v3/projects/24fb6ed68f4644c284eee691a5ef71e0` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:24.636426 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8079e74d-f0b0-4bb9-887e-5a716244e047 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24fb6ed68f4644c284eee691a5ef71e0)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:24.637095 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65119f20-2885-484d-98e6-f0d4e167783c admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:24.637781 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8079e74d-f0b0-4bb9-887e-5a716244e047 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:24.652605 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65119f20-2885-484d-98e6-f0d4e167783c admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:24.658008 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea68bf2-d4ad-49ce-85f7-295c38d32b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:24.658623 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea68bf2-d4ad-49ce-85f7-295c38d32b6d admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:24.658909 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea68bf2-d4ad-49ce-85f7-295c38d32b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:24.659224 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea68bf2-d4ad-49ce-85f7-295c38d32b6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:24.659968 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea68bf2-d4ad-49ce-85f7-295c38d32b6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:24.661600 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea68bf2-d4ad-49ce-85f7-295c38d32b6d admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:24.673680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1045/2101] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:24.677639 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea68bf2-d4ad-49ce-85f7-295c38d32b6d admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:24.688273 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea68bf2-d4ad-49ce-85f7-295c38d32b6d admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:24.689576 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-8079e74d-f0b0-4bb9-887e-5a716244e047 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24fb6ed68f4644c284eee691a5ef71e0', 'request_id': 'req-8079e74d-f0b0-4bb9-887e-5a716244e047'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:24.695350 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-8079e74d-f0b0-4bb9-887e-5a716244e047 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24fb6ed68f4644c284eee691a5ef71e0', 'request_id': 'req-8079e74d-f0b0-4bb9-887e-5a716244e047'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:24.703237 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-8079e74d-f0b0-4bb9-887e-5a716244e047 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24fb6ed68f4644c284eee691a5ef71e0', 'request_id': 'req-8079e74d-f0b0-4bb9-887e-5a716244e047'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:24.716936 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-8079e74d-f0b0-4bb9-887e-5a716244e047 admin admin] The token cache is being invalidate because project 24fb6ed68f4644c284eee691a5ef71e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:43:24.717241 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-8079e74d-f0b0-4bb9-887e-5a716244e047 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:24.717885 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-8079e74d-f0b0-4bb9-887e-5a716244e047 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:24.719420 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1057/2102] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:43:24 2022] DELETE /identity/v3/projects/24fb6ed68f4644c284eee691a5ef71e0 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:24.723298 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8079e74d-f0b0-4bb9-887e-5a716244e047 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:24.724686 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-8079e74d-f0b0-4bb9-887e-5a716244e047 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:24.750268 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6b3a12-8653-450c-a0a0-f79105ab652d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:24.750996 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b3a12-8653-450c-a0a0-f79105ab652d admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:24.751317 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b3a12-8653-450c-a0a0-f79105ab652d admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:24.751655 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b3a12-8653-450c-a0a0-f79105ab652d admin admin] PATH_INFO: `/v3/projects/3305e0e019484954ad9ae930b09e6849/users/d31de227c57643839584038c5bb8873a/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:24.766792 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6b3a12-8653-450c-a0a0-f79105ab652d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3305e0e019484954ad9ae930b09e6849, user_id=d31de227c57643839584038c5bb8873a, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:24.769454 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6b3a12-8653-450c-a0a0-f79105ab652d admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:24.783536 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1046/2103] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:24 2022] PUT /identity/v3/projects/3305e0e019484954ad9ae930b09e6849/users/d31de227c57643839584038c5bb8873a/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:43:24.787298 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6b3a12-8653-450c-a0a0-f79105ab652d admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:24.789222 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-ce6b3a12-8653-450c-a0a0-f79105ab652d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:24.792525 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8079e74d-f0b0-4bb9-887e-5a716244e047 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:24.805792 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:24.860704 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6b3a12-8653-450c-a0a0-f79105ab652d admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:24.865934 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edee239-c97f-4286-817c-c39c7943e618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:24.866537 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edee239-c97f-4286-817c-c39c7943e618 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:24.866882 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edee239-c97f-4286-817c-c39c7943e618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:24.867172 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edee239-c97f-4286-817c-c39c7943e618 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:24.867837 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edee239-c97f-4286-817c-c39c7943e618 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:24.868099 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb78f4d2-40f9-4db8-9b50-10a7fa054cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:24.868745 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78f4d2-40f9-4db8-9b50-10a7fa054cc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:24.869027 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78f4d2-40f9-4db8-9b50-10a7fa054cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:24.869305 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78f4d2-40f9-4db8-9b50-10a7fa054cc1 admin admin] PATH_INFO: `/v3/users/6e20b84b0b1f43d4877b38d5a0bc6a60` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:24.869544 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edee239-c97f-4286-817c-c39c7943e618 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:24.870470 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb78f4d2-40f9-4db8-9b50-10a7fa054cc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e20b84b0b1f43d4877b38d5a0bc6a60)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:24.873114 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb78f4d2-40f9-4db8-9b50-10a7fa054cc1 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:24.881218 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1047/2104] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:24 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:43:24.885335 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edee239-c97f-4286-817c-c39c7943e618 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:24.895952 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edee239-c97f-4286-817c-c39c7943e618 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:24.902582 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263720fe-867e-425b-a4df-f416b0882564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:24.907703 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263720fe-867e-425b-a4df-f416b0882564 admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:24.908497 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263720fe-867e-425b-a4df-f416b0882564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:24.908497 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263720fe-867e-425b-a4df-f416b0882564 admin admin] PATH_INFO: `/v3/projects/3305e0e019484954ad9ae930b09e6849/users/d31de227c57643839584038c5bb8873a/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:24.911362 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263720fe-867e-425b-a4df-f416b0882564 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3305e0e019484954ad9ae930b09e6849, user_id=d31de227c57643839584038c5bb8873a, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:24.914485 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263720fe-867e-425b-a4df-f416b0882564 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:24.939599 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1048/2105] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:24 2022] PUT /identity/v3/projects/3305e0e019484954ad9ae930b09e6849/users/d31de227c57643839584038c5bb8873a/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:24.939782 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-cb78f4d2-40f9-4db8-9b50-10a7fa054cc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e20b84b0b1f43d4877b38d5a0bc6a60'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:24.940780 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-cb78f4d2-40f9-4db8-9b50-10a7fa054cc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e20b84b0b1f43d4877b38d5a0bc6a60'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:24.941435 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a6075a-1613-449f-90c2-094d4dd85899 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:24.941721 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a6075a-1613-449f-90c2-094d4dd85899 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:24.941991 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a6075a-1613-449f-90c2-094d4dd85899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:24.949480 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-cb78f4d2-40f9-4db8-9b50-10a7fa054cc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e20b84b0b1f43d4877b38d5a0bc6a60'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:24.958377 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-cb78f4d2-40f9-4db8-9b50-10a7fa054cc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e20b84b0b1f43d4877b38d5a0bc6a60'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:24.959187 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-cb78f4d2-40f9-4db8-9b50-10a7fa054cc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e20b84b0b1f43d4877b38d5a0bc6a60'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:24.985948 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-03a6075a-1613-449f-90c2-094d4dd85899 None None] MFA Rules not processed for user `d31de227c57643839584038c5bb8873a`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:24.986484 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1058/2106] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:43:24 2022] DELETE /identity/v3/users/6e20b84b0b1f43d4877b38d5a0bc6a60 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:24.988271 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-03a6075a-1613-449f-90c2-094d4dd85899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:25.020869 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:25.095242 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb78f4d2-40f9-4db8-9b50-10a7fa054cc1 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:25.095621 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1049/2107] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:24 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:25.097300 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-cb78f4d2-40f9-4db8-9b50-10a7fa054cc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:25.106667 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a6075a-1613-449f-90c2-094d4dd85899 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:25.109043 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-03a6075a-1613-449f-90c2-094d4dd85899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:25.183045 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a6075a-1613-449f-90c2-094d4dd85899 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:25.188466 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a3c96f-b159-4926-a64f-c29dbec94ef7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:25.189104 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a3c96f-b159-4926-a64f-c29dbec94ef7 service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:25.189983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a3c96f-b159-4926-a64f-c29dbec94ef7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:25.190625 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a3c96f-b159-4926-a64f-c29dbec94ef7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:25.193696 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb78f4d2-40f9-4db8-9b50-10a7fa054cc1 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:25.200010 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3983135c-cdf4-43ab-9e35-c3471f5ec216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:25.200933 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983135c-cdf4-43ab-9e35-c3471f5ec216 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:25.201307 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983135c-cdf4-43ab-9e35-c3471f5ec216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:25.201669 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983135c-cdf4-43ab-9e35-c3471f5ec216 admin admin] PATH_INFO: `/v3/users/bd0fb5e4286842e8acc424eb4163c2cd` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:25.202902 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3983135c-cdf4-43ab-9e35-c3471f5ec216 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd0fb5e4286842e8acc424eb4163c2cd)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:25.204337 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a3c96f-b159-4926-a64f-c29dbec94ef7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:25.205312 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3983135c-cdf4-43ab-9e35-c3471f5ec216 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:25.206358 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a3c96f-b159-4926-a64f-c29dbec94ef7 service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:25.221238 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1050/2108] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:43:25 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:43:25.266782 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:25.278936 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-3983135c-cdf4-43ab-9e35-c3471f5ec216 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd0fb5e4286842e8acc424eb4163c2cd'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:25.279774 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-3983135c-cdf4-43ab-9e35-c3471f5ec216 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd0fb5e4286842e8acc424eb4163c2cd'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:25.289530 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-3983135c-cdf4-43ab-9e35-c3471f5ec216 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd0fb5e4286842e8acc424eb4163c2cd'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:25.311397 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-3983135c-cdf4-43ab-9e35-c3471f5ec216 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd0fb5e4286842e8acc424eb4163c2cd'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:25.311397 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-3983135c-cdf4-43ab-9e35-c3471f5ec216 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd0fb5e4286842e8acc424eb4163c2cd'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:25.312585 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a3c96f-b159-4926-a64f-c29dbec94ef7 service cinder] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:25.314228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-47a3c96f-b159-4926-a64f-c29dbec94ef7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:25.336440 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a3c96f-b159-4926-a64f-c29dbec94ef7 service cinder] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:25.341422 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1059/2109] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:43:25 2022] DELETE /identity/v3/users/bd0fb5e4286842e8acc424eb4163c2cd => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:43:25.342542 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f4ad90-1c5e-4b56-a119-880881db1a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:25.343188 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3983135c-cdf4-43ab-9e35-c3471f5ec216 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:25.343329 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f4ad90-1c5e-4b56-a119-880881db1a0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:25.343694 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f4ad90-1c5e-4b56-a119-880881db1a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:25.344162 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f4ad90-1c5e-4b56-a119-880881db1a0c admin admin] PATH_INFO: `/v3/projects/dde4ff1a92214091bb54938b30b1dac0` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:25.345365 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-3983135c-cdf4-43ab-9e35-c3471f5ec216 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:25.345574 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f4ad90-1c5e-4b56-a119-880881db1a0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=dde4ff1a92214091bb54938b30b1dac0)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:25.347558 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f4ad90-1c5e-4b56-a119-880881db1a0c admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:25.397297 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-c7f4ad90-1c5e-4b56-a119-880881db1a0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dde4ff1a92214091bb54938b30b1dac0', 'request_id': 'req-c7f4ad90-1c5e-4b56-a119-880881db1a0c'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:25.405866 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-c7f4ad90-1c5e-4b56-a119-880881db1a0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dde4ff1a92214091bb54938b30b1dac0', 'request_id': 'req-c7f4ad90-1c5e-4b56-a119-880881db1a0c'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:25.412644 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3983135c-cdf4-43ab-9e35-c3471f5ec216 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:25.413742 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-c7f4ad90-1c5e-4b56-a119-880881db1a0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dde4ff1a92214091bb54938b30b1dac0', 'request_id': 'req-c7f4ad90-1c5e-4b56-a119-880881db1a0c'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:25.426935 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-c7f4ad90-1c5e-4b56-a119-880881db1a0c admin admin] The token cache is being invalidate because project dde4ff1a92214091bb54938b30b1dac0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:43:25.427259 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-c7f4ad90-1c5e-4b56-a119-880881db1a0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:25.427930 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-c7f4ad90-1c5e-4b56-a119-880881db1a0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:25.429886 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1051/2110] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:43:25 2022] DELETE /identity/v3/projects/dde4ff1a92214091bb54938b30b1dac0 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:43:25.431067 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a1e65-3517-432d-8029-056be786aa0b None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:25.431394 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a1e65-3517-432d-8029-056be786aa0b None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:25.431670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a1e65-3517-432d-8029-056be786aa0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:25.460289 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-0d3a1e65-3517-432d-8029-056be786aa0b None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:25.461517 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-0d3a1e65-3517-432d-8029-056be786aa0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:25.471970 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b252282-e87e-4620-8d61-16570075f3fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:25.472775 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b252282-e87e-4620-8d61-16570075f3fc service cinder] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:25.473301 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b252282-e87e-4620-8d61-16570075f3fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:25.473687 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b252282-e87e-4620-8d61-16570075f3fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:25.476686 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-8b252282-e87e-4620-8d61-16570075f3fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:25.508652 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:25.527510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1052/2111] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:25.530096 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca706b-ca6b-439a-aa4c-25ecd201186b None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:25.530669 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca706b-ca6b-439a-aa4c-25ecd201186b None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:25.530949 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca706b-ca6b-439a-aa4c-25ecd201186b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:25.544542 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b252282-e87e-4620-8d61-16570075f3fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:25.545977 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b252282-e87e-4620-8d61-16570075f3fc service cinder] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:25.550614 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-e4ca706b-ca6b-439a-aa4c-25ecd201186b None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:25.551877 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-e4ca706b-ca6b-439a-aa4c-25ecd201186b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:25.555101 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1060/2112] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:43:25 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:43:25.555935 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b252282-e87e-4620-8d61-16570075f3fc service cinder] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:25.557283 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-8b252282-e87e-4620-8d61-16570075f3fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:25.561069 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1053/2113] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:25.569684 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887314a-beee-4834-beef-86183b76c694 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:25.569967 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887314a-beee-4834-beef-86183b76c694 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:25.570264 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887314a-beee-4834-beef-86183b76c694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:25.589896 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:25.590571 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-3887314a-beee-4834-beef-86183b76c694 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:25.593504 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-3887314a-beee-4834-beef-86183b76c694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:25.604615 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1054/2114] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:43:25.610241 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887314a-beee-4834-beef-86183b76c694 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:25.622436 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887314a-beee-4834-beef-86183b76c694 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:25.623157 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b252282-e87e-4620-8d61-16570075f3fc service cinder] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:25.627988 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d0082c-57d1-482f-8cab-762bb83f36ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:25.629356 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0082c-57d1-482f-8cab-762bb83f36ff service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:25.629585 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283751a3-5799-4d9e-80a6-22c41d93007f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:25.630258 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0082c-57d1-482f-8cab-762bb83f36ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:25.630375 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283751a3-5799-4d9e-80a6-22c41d93007f admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:25.630525 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0082c-57d1-482f-8cab-762bb83f36ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:25.630749 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283751a3-5799-4d9e-80a6-22c41d93007f admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:25.631127 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283751a3-5799-4d9e-80a6-22c41d93007f admin admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:25.632093 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283751a3-5799-4d9e-80a6-22c41d93007f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:43:25.632556 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283751a3-5799-4d9e-80a6-22c41d93007f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:25.634358 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283751a3-5799-4d9e-80a6-22c41d93007f admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:25.644294 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d0082c-57d1-482f-8cab-762bb83f36ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:25.646575 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d0082c-57d1-482f-8cab-762bb83f36ff service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:25.649857 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1055/2115] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 14:43:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:43:25.654590 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283751a3-5799-4d9e-80a6-22c41d93007f admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:25.664850 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283751a3-5799-4d9e-80a6-22c41d93007f admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:25.670671 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1061/2116] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:43:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:43:25.676354 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baedb96-8ce7-4dc1-82b2-8a757c060e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:25.677046 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baedb96-8ce7-4dc1-82b2-8a757c060e27 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:25.677392 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baedb96-8ce7-4dc1-82b2-8a757c060e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:25.678000 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baedb96-8ce7-4dc1-82b2-8a757c060e27 admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:25.679240 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baedb96-8ce7-4dc1-82b2-8a757c060e27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:25.683373 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baedb96-8ce7-4dc1-82b2-8a757c060e27 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:25.716721 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1056/2117] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:43:25 2022] POST /identity/v3/projects => generated 385 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:43:25.721942 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d0082c-57d1-482f-8cab-762bb83f36ff service neutron] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:25.737250 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d0082c-57d1-482f-8cab-762bb83f36ff service neutron] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:25.809297 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7115acb9-7a64-4857-b6be-e2124121c162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:25.810333 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115acb9-7a64-4857-b6be-e2124121c162 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:25.811399 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115acb9-7a64-4857-b6be-e2124121c162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:25.811399 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115acb9-7a64-4857-b6be-e2124121c162 admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:25.812283 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7115acb9-7a64-4857-b6be-e2124121c162 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:25.813956 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7115acb9-7a64-4857-b6be-e2124121c162 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:25.846384 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1062/2118] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:43:25 2022] POST /identity/v3/users => generated 293 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:25.848025 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baedb96-8ce7-4dc1-82b2-8a757c060e27 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:25.861486 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baedb96-8ce7-4dc1-82b2-8a757c060e27 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:25.870675 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfa9f5a-60b9-428c-bd9b-1bd88a4b8e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:25.870675 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa9f5a-60b9-428c-bd9b-1bd88a4b8e98 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:25.870675 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa9f5a-60b9-428c-bd9b-1bd88a4b8e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:25.870675 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa9f5a-60b9-428c-bd9b-1bd88a4b8e98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:25.871173 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfa9f5a-60b9-428c-bd9b-1bd88a4b8e98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:25.871173 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfa9f5a-60b9-428c-bd9b-1bd88a4b8e98 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:25.884186 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1057/2119] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:25.888851 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7115acb9-7a64-4857-b6be-e2124121c162 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:25.911237 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7115acb9-7a64-4857-b6be-e2124121c162 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:25.925525 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b50b0f7-c7b3-434a-9ce3-a6ea10af5f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:25.926155 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b50b0f7-c7b3-434a-9ce3-a6ea10af5f20 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:25.926430 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b50b0f7-c7b3-434a-9ce3-a6ea10af5f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:25.926703 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b50b0f7-c7b3-434a-9ce3-a6ea10af5f20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:25.927861 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b50b0f7-c7b3-434a-9ce3-a6ea10af5f20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:25.929479 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b50b0f7-c7b3-434a-9ce3-a6ea10af5f20 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:25.963514 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1063/2120] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:25 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:25.968546 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfa9f5a-60b9-428c-bd9b-1bd88a4b8e98 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:25.995663 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfa9f5a-60b9-428c-bd9b-1bd88a4b8e98 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:26.008835 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62452485-74b9-461f-85dc-1fa7e60cf99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:26.010474 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62452485-74b9-461f-85dc-1fa7e60cf99a admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:26.010755 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62452485-74b9-461f-85dc-1fa7e60cf99a admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:26.011029 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62452485-74b9-461f-85dc-1fa7e60cf99a admin admin] PATH_INFO: `/v3/projects/54444cd4cd134a59bec2232c26310a28/users/8dfa19ff3652461d82c4d773304aee67/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:26.043536 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62452485-74b9-461f-85dc-1fa7e60cf99a admin admin] RBAC: Authorizing `identity:create_grant(project_id=54444cd4cd134a59bec2232c26310a28, user_id=8dfa19ff3652461d82c4d773304aee67, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:26.046973 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62452485-74b9-461f-85dc-1fa7e60cf99a admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:26.067252 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b50b0f7-c7b3-434a-9ce3-a6ea10af5f20 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:26.070992 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-0b50b0f7-c7b3-434a-9ce3-a6ea10af5f20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:26.079848 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1058/2121] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:25 2022] PUT /identity/v3/projects/54444cd4cd134a59bec2232c26310a28/users/8dfa19ff3652461d82c4d773304aee67/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:26.087398 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b50b0f7-c7b3-434a-9ce3-a6ea10af5f20 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:26.093977 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250524e3-3467-4d57-a320-a171cd60db67 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:26.094381 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250524e3-3467-4d57-a320-a171cd60db67 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:26.094771 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250524e3-3467-4d57-a320-a171cd60db67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:26.140822 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-250524e3-3467-4d57-a320-a171cd60db67 None None] MFA Rules not processed for user `8dfa19ff3652461d82c4d773304aee67`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:26.142054 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-250524e3-3467-4d57-a320-a171cd60db67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:26.164150 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71662686-52fc-4d0b-9237-788ff4afb2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:26.164795 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71662686-52fc-4d0b-9237-788ff4afb2f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:26.165075 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71662686-52fc-4d0b-9237-788ff4afb2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:26.165370 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71662686-52fc-4d0b-9237-788ff4afb2f6 admin admin] PATH_INFO: `/v3/projects/9f437fc48ab748c0bec905e55642deaa` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:26.166595 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71662686-52fc-4d0b-9237-788ff4afb2f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f437fc48ab748c0bec905e55642deaa)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:26.168537 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71662686-52fc-4d0b-9237-788ff4afb2f6 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:26.221554 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1059/2122] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:26.223659 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-71662686-52fc-4d0b-9237-788ff4afb2f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f437fc48ab748c0bec905e55642deaa', 'request_id': 'req-71662686-52fc-4d0b-9237-788ff4afb2f6'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:26.225483 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:26.227591 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250524e3-3467-4d57-a320-a171cd60db67 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:26.236495 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-71662686-52fc-4d0b-9237-788ff4afb2f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f437fc48ab748c0bec905e55642deaa', 'request_id': 'req-71662686-52fc-4d0b-9237-788ff4afb2f6'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:26.246327 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-71662686-52fc-4d0b-9237-788ff4afb2f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f437fc48ab748c0bec905e55642deaa', 'request_id': 'req-71662686-52fc-4d0b-9237-788ff4afb2f6'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:26.249699 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250524e3-3467-4d57-a320-a171cd60db67 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:26.259651 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-71662686-52fc-4d0b-9237-788ff4afb2f6 admin admin] The token cache is being invalidate because project 9f437fc48ab748c0bec905e55642deaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:43:26.259944 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-71662686-52fc-4d0b-9237-788ff4afb2f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:26.260584 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-71662686-52fc-4d0b-9237-788ff4afb2f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:26.262112 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1064/2123] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:43:26 2022] DELETE /identity/v3/projects/9f437fc48ab748c0bec905e55642deaa => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:26.266452 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71662686-52fc-4d0b-9237-788ff4afb2f6 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:26.267849 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-71662686-52fc-4d0b-9237-788ff4afb2f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:26.302197 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:26.311465 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac4a3b-f1f4-42ad-befe-dc04ccad1318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:26.312073 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac4a3b-f1f4-42ad-befe-dc04ccad1318 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:26.312349 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac4a3b-f1f4-42ad-befe-dc04ccad1318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:26.312648 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac4a3b-f1f4-42ad-befe-dc04ccad1318 admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:26.313276 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bac4a3b-f1f4-42ad-befe-dc04ccad1318 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:26.315759 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bac4a3b-f1f4-42ad-befe-dc04ccad1318 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:26.332379 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71662686-52fc-4d0b-9237-788ff4afb2f6 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:26.338156 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccd0cc7-813a-475d-a1f7-48cf24360da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:26.338783 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd0cc7-813a-475d-a1f7-48cf24360da0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:26.339077 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd0cc7-813a-475d-a1f7-48cf24360da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:26.339403 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd0cc7-813a-475d-a1f7-48cf24360da0 admin admin] PATH_INFO: `/v3/users/9130b4bf347b402ca2070f52b4b5b9c4` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:26.340520 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccd0cc7-813a-475d-a1f7-48cf24360da0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9130b4bf347b402ca2070f52b4b5b9c4)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:26.343004 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccd0cc7-813a-475d-a1f7-48cf24360da0 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:26.344345 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1060/2124] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:43:26 2022] POST /identity/v3/projects => generated 385 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:43:26.348522 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac4a3b-f1f4-42ad-befe-dc04ccad1318 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:26.350828 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-3bac4a3b-f1f4-42ad-befe-dc04ccad1318 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:26.392338 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-8ccd0cc7-813a-475d-a1f7-48cf24360da0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9130b4bf347b402ca2070f52b4b5b9c4'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:26.393008 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-8ccd0cc7-813a-475d-a1f7-48cf24360da0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9130b4bf347b402ca2070f52b4b5b9c4'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:26.400542 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-8ccd0cc7-813a-475d-a1f7-48cf24360da0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9130b4bf347b402ca2070f52b4b5b9c4'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:26.409885 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-8ccd0cc7-813a-475d-a1f7-48cf24360da0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9130b4bf347b402ca2070f52b4b5b9c4'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:26.410497 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-8ccd0cc7-813a-475d-a1f7-48cf24360da0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9130b4bf347b402ca2070f52b4b5b9c4'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:26.420026 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac4a3b-f1f4-42ad-befe-dc04ccad1318 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:26.424731 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3e7ad-de4b-4848-adaa-821f6193e4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:26.425334 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3e7ad-de4b-4848-adaa-821f6193e4b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:26.425725 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3e7ad-de4b-4848-adaa-821f6193e4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:26.426005 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3e7ad-de4b-4848-adaa-821f6193e4b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:26.426636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e3e7ad-de4b-4848-adaa-821f6193e4b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:26.428623 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e3e7ad-de4b-4848-adaa-821f6193e4b3 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:26.428859 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1065/2125] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:43:26 2022] DELETE /identity/v3/users/9130b4bf347b402ca2070f52b4b5b9c4 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:43:26.452563 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1061/2126] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:43:26 2022] POST /identity/v3/users => generated 299 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:26.456481 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccd0cc7-813a-475d-a1f7-48cf24360da0 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:26.458006 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-8ccd0cc7-813a-475d-a1f7-48cf24360da0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:26.526897 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:26.531751 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccd0cc7-813a-475d-a1f7-48cf24360da0 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:26.538620 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607537fa-5a1b-4747-96c5-819cd1885371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:26.539734 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607537fa-5a1b-4747-96c5-819cd1885371 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:26.540272 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607537fa-5a1b-4747-96c5-819cd1885371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:26.540854 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607537fa-5a1b-4747-96c5-819cd1885371 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:26.541754 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607537fa-5a1b-4747-96c5-819cd1885371 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:26.543792 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607537fa-5a1b-4747-96c5-819cd1885371 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:26.554599 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1066/2127] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:26 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:26.558580 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3e7ad-de4b-4848-adaa-821f6193e4b3 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:26.569967 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3e7ad-de4b-4848-adaa-821f6193e4b3 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:26.574898 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ff31f-0a94-45a0-aef0-35d7f87843fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:26.575842 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ff31f-0a94-45a0-aef0-35d7f87843fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:26.576205 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ff31f-0a94-45a0-aef0-35d7f87843fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:26.576552 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ff31f-0a94-45a0-aef0-35d7f87843fa admin admin] PATH_INFO: `/v3/projects/8a1039d4f2324fcc82415d52d04bc527/users/a650b7c058a84a0885f002d78dd90c66/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:26.600132 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85ff31f-0a94-45a0-aef0-35d7f87843fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a1039d4f2324fcc82415d52d04bc527, user_id=a650b7c058a84a0885f002d78dd90c66, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:26.602802 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85ff31f-0a94-45a0-aef0-35d7f87843fa admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:26.617201 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1062/2128] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:26 2022] PUT /identity/v3/projects/8a1039d4f2324fcc82415d52d04bc527/users/a650b7c058a84a0885f002d78dd90c66/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:26.621346 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607537fa-5a1b-4747-96c5-819cd1885371 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:26.632916 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607537fa-5a1b-4747-96c5-819cd1885371 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:26.692409 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4441580-611b-4084-a737-38dccc7d892a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:26.693808 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4441580-611b-4084-a737-38dccc7d892a admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:26.693808 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4441580-611b-4084-a737-38dccc7d892a admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:26.693808 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4441580-611b-4084-a737-38dccc7d892a admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:26.694190 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4441580-611b-4084-a737-38dccc7d892a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:26.696121 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4441580-611b-4084-a737-38dccc7d892a admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:26.707891 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1067/2129] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:26 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:26.712177 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ff31f-0a94-45a0-aef0-35d7f87843fa admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:26.724096 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4441580-611b-4084-a737-38dccc7d892a admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:26.726710 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-b4441580-611b-4084-a737-38dccc7d892a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:26.728431 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ff31f-0a94-45a0-aef0-35d7f87843fa admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:26.738796 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91670bbf-be32-4722-adad-33e8eee8227d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:26.739787 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91670bbf-be32-4722-adad-33e8eee8227d admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:26.740075 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91670bbf-be32-4722-adad-33e8eee8227d admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:26.740351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91670bbf-be32-4722-adad-33e8eee8227d admin admin] PATH_INFO: `/v3/projects/8a1039d4f2324fcc82415d52d04bc527/users/a650b7c058a84a0885f002d78dd90c66/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:26.740464 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4441580-611b-4084-a737-38dccc7d892a admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:26.742754 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91670bbf-be32-4722-adad-33e8eee8227d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a1039d4f2324fcc82415d52d04bc527, user_id=a650b7c058a84a0885f002d78dd90c66, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:26.745093 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0839e10-40dd-41fe-991b-4c90b21b5043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:26.745460 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91670bbf-be32-4722-adad-33e8eee8227d admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:26.745741 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0839e10-40dd-41fe-991b-4c90b21b5043 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:26.746014 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0839e10-40dd-41fe-991b-4c90b21b5043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:26.746283 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0839e10-40dd-41fe-991b-4c90b21b5043 admin admin] PATH_INFO: `/v3/projects/dda573e027aa4d90bca096c157d14fdc` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:26.748040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0839e10-40dd-41fe-991b-4c90b21b5043 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dda573e027aa4d90bca096c157d14fdc)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:26.750863 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0839e10-40dd-41fe-991b-4c90b21b5043 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:26.757938 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1063/2130] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:26 2022] PUT /identity/v3/projects/8a1039d4f2324fcc82415d52d04bc527/users/a650b7c058a84a0885f002d78dd90c66/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:26.770036 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250cbb0-2ef6-4fa0-b008-fb146a30e970 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:26.770635 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250cbb0-2ef6-4fa0-b008-fb146a30e970 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:26.770743 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250cbb0-2ef6-4fa0-b008-fb146a30e970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:26.799135 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-0250cbb0-2ef6-4fa0-b008-fb146a30e970 None None] MFA Rules not processed for user `a650b7c058a84a0885f002d78dd90c66`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:26.800855 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-0250cbb0-2ef6-4fa0-b008-fb146a30e970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:26.801126 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-e0839e10-40dd-41fe-991b-4c90b21b5043 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dda573e027aa4d90bca096c157d14fdc', 'request_id': 'req-e0839e10-40dd-41fe-991b-4c90b21b5043'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:26.807187 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-e0839e10-40dd-41fe-991b-4c90b21b5043 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dda573e027aa4d90bca096c157d14fdc', 'request_id': 'req-e0839e10-40dd-41fe-991b-4c90b21b5043'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:26.815001 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-e0839e10-40dd-41fe-991b-4c90b21b5043 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dda573e027aa4d90bca096c157d14fdc', 'request_id': 'req-e0839e10-40dd-41fe-991b-4c90b21b5043'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:26.829089 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-e0839e10-40dd-41fe-991b-4c90b21b5043 admin admin] The token cache is being invalidate because project dda573e027aa4d90bca096c157d14fdc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:43:26.829719 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-e0839e10-40dd-41fe-991b-4c90b21b5043 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:26.830045 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-e0839e10-40dd-41fe-991b-4c90b21b5043 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:26.831616 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1068/2131] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:43:26 2022] DELETE /identity/v3/projects/dda573e027aa4d90bca096c157d14fdc => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:43:26.835382 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0839e10-40dd-41fe-991b-4c90b21b5043 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:26.836706 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-e0839e10-40dd-41fe-991b-4c90b21b5043 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:26.876080 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1064/2132] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:26 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:26.887541 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0250cbb0-2ef6-4fa0-b008-fb146a30e970 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:26.889684 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-0250cbb0-2ef6-4fa0-b008-fb146a30e970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:26.906051 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0839e10-40dd-41fe-991b-4c90b21b5043 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:26.910475 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:26.910710 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ef883-1c8d-439b-8473-52e0e9ef2145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:26.911474 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ef883-1c8d-439b-8473-52e0e9ef2145 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:26.911645 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ef883-1c8d-439b-8473-52e0e9ef2145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:26.911923 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ef883-1c8d-439b-8473-52e0e9ef2145 admin admin] PATH_INFO: `/v3/users/bbea3303bbab4a1dad81339d5b446ba4` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:26.913380 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8ef883-1c8d-439b-8473-52e0e9ef2145 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbea3303bbab4a1dad81339d5b446ba4)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:26.915240 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8ef883-1c8d-439b-8473-52e0e9ef2145 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:26.929578 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:26.951636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0250cbb0-2ef6-4fa0-b008-fb146a30e970 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:26.955375 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b711ca9f-045e-4aae-978c-e15b6ce13fbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:26.955983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b711ca9f-045e-4aae-978c-e15b6ce13fbc service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:26.956262 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b711ca9f-045e-4aae-978c-e15b6ce13fbc service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:26.956535 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b711ca9f-045e-4aae-978c-e15b6ce13fbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:26.958312 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-b711ca9f-045e-4aae-978c-e15b6ce13fbc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:26.964288 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-0e8ef883-1c8d-439b-8473-52e0e9ef2145 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbea3303bbab4a1dad81339d5b446ba4'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:26.964974 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-0e8ef883-1c8d-439b-8473-52e0e9ef2145 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbea3303bbab4a1dad81339d5b446ba4'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:26.975625 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-0e8ef883-1c8d-439b-8473-52e0e9ef2145 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbea3303bbab4a1dad81339d5b446ba4'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:26.984301 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-0e8ef883-1c8d-439b-8473-52e0e9ef2145 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbea3303bbab4a1dad81339d5b446ba4'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:26.985105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-0e8ef883-1c8d-439b-8473-52e0e9ef2145 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbea3303bbab4a1dad81339d5b446ba4'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:27.007503 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1069/2133] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:43:26 2022] DELETE /identity/v3/users/bbea3303bbab4a1dad81339d5b446ba4 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:43:27.034041 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b711ca9f-045e-4aae-978c-e15b6ce13fbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:27.035522 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b711ca9f-045e-4aae-978c-e15b6ce13fbc service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:27.045103 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1065/2134] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:43:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:43:27.311442 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ef883-1c8d-439b-8473-52e0e9ef2145 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:27.313154 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-0e8ef883-1c8d-439b-8473-52e0e9ef2145 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:27.394807 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ef883-1c8d-439b-8473-52e0e9ef2145 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:27.401466 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e577ce31-73f2-41bf-a95f-77804b7a7c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:27.403110 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577ce31-73f2-41bf-a95f-77804b7a7c3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:27.404235 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577ce31-73f2-41bf-a95f-77804b7a7c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:27.404724 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577ce31-73f2-41bf-a95f-77804b7a7c3e admin admin] PATH_INFO: `/v3/projects/144e2687b09a4449a0583c3bab7362b3` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:27.406082 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e577ce31-73f2-41bf-a95f-77804b7a7c3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=144e2687b09a4449a0583c3bab7362b3)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:27.408154 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e577ce31-73f2-41bf-a95f-77804b7a7c3e admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:27.470150 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-e577ce31-73f2-41bf-a95f-77804b7a7c3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '144e2687b09a4449a0583c3bab7362b3', 'request_id': 'req-e577ce31-73f2-41bf-a95f-77804b7a7c3e'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:27.479486 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-e577ce31-73f2-41bf-a95f-77804b7a7c3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '144e2687b09a4449a0583c3bab7362b3', 'request_id': 'req-e577ce31-73f2-41bf-a95f-77804b7a7c3e'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:27.487422 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-e577ce31-73f2-41bf-a95f-77804b7a7c3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '144e2687b09a4449a0583c3bab7362b3', 'request_id': 'req-e577ce31-73f2-41bf-a95f-77804b7a7c3e'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:27.501496 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-e577ce31-73f2-41bf-a95f-77804b7a7c3e admin admin] The token cache is being invalidate because project 144e2687b09a4449a0583c3bab7362b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:43:27.501869 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-e577ce31-73f2-41bf-a95f-77804b7a7c3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:27.502506 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-e577ce31-73f2-41bf-a95f-77804b7a7c3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:27.504594 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1070/2135] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:43:27 2022] DELETE /identity/v3/projects/144e2687b09a4449a0583c3bab7362b3 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:43:27.509390 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e2521-9c3a-4d0c-940e-dae503beb885 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:27.509679 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e2521-9c3a-4d0c-940e-dae503beb885 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:27.509954 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e2521-9c3a-4d0c-940e-dae503beb885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:27.532545 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-552e2521-9c3a-4d0c-940e-dae503beb885 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:27.533779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-552e2521-9c3a-4d0c-940e-dae503beb885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:27.592648 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1066/2136] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:27.601465 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895169a6-8984-4ebe-8153-8f0594cd5804 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:27.601749 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895169a6-8984-4ebe-8153-8f0594cd5804 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:27.602024 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895169a6-8984-4ebe-8153-8f0594cd5804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:27.622217 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-895169a6-8984-4ebe-8153-8f0594cd5804 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:27.623491 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-895169a6-8984-4ebe-8153-8f0594cd5804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:27.631426 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1071/2137] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:27.635548 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e2521-9c3a-4d0c-940e-dae503beb885 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:27.646300 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e2521-9c3a-4d0c-940e-dae503beb885 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:27.650939 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a4d373-e7dd-4d21-90bb-44c42409a4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:27.651597 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a4d373-e7dd-4d21-90bb-44c42409a4e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:27.651875 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a4d373-e7dd-4d21-90bb-44c42409a4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:27.652147 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a4d373-e7dd-4d21-90bb-44c42409a4e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:27.652791 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a4d373-e7dd-4d21-90bb-44c42409a4e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:43:27.653091 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a4d373-e7dd-4d21-90bb-44c42409a4e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:27.655632 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a4d373-e7dd-4d21-90bb-44c42409a4e3 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:27.666972 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1067/2138] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 14:43:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:43:27.671164 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895169a6-8984-4ebe-8153-8f0594cd5804 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:27.680431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895169a6-8984-4ebe-8153-8f0594cd5804 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:27.685301 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5bf71-f387-4409-86ee-0378059925db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:27.685895 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5bf71-f387-4409-86ee-0378059925db admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:27.686172 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5bf71-f387-4409-86ee-0378059925db admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:27.686440 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5bf71-f387-4409-86ee-0378059925db admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:27.687062 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee5bf71-f387-4409-86ee-0378059925db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:27.688991 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee5bf71-f387-4409-86ee-0378059925db admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:27.704435 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:27.719711 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1072/2139] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:43:27 2022] POST /identity/v3/projects => generated 363 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:27.723835 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a4d373-e7dd-4d21-90bb-44c42409a4e3 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:27.734069 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a4d373-e7dd-4d21-90bb-44c42409a4e3 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:27.790572 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190b965a-6926-4eda-900d-980b4f9fb149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:27.791168 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b965a-6926-4eda-900d-980b4f9fb149 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:27.791482 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b965a-6926-4eda-900d-980b4f9fb149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:27.791758 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b965a-6926-4eda-900d-980b4f9fb149 admin admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:27.792389 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190b965a-6926-4eda-900d-980b4f9fb149 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:27.794343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190b965a-6926-4eda-900d-980b4f9fb149 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:27.802241 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:27.816674 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1068/2140] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:43:27 2022] POST /identity/v3/users => generated 282 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:27.820993 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5bf71-f387-4409-86ee-0378059925db admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:27.831974 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5bf71-f387-4409-86ee-0378059925db admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:27.836702 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6284108-acad-4fd0-b8c7-588a54e0eb0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:27.837299 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6284108-acad-4fd0-b8c7-588a54e0eb0f admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:27.837586 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6284108-acad-4fd0-b8c7-588a54e0eb0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:27.837856 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6284108-acad-4fd0-b8c7-588a54e0eb0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:27.838498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6284108-acad-4fd0-b8c7-588a54e0eb0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:27.840126 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6284108-acad-4fd0-b8c7-588a54e0eb0f admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:27.850513 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1073/2141] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:27.854301 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190b965a-6926-4eda-900d-980b4f9fb149 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:27.865477 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190b965a-6926-4eda-900d-980b4f9fb149 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:27.870104 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889f9a80-33d9-4fd0-8064-23f6f4f18139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:27.870692 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f9a80-33d9-4fd0-8064-23f6f4f18139 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:27.870969 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f9a80-33d9-4fd0-8064-23f6f4f18139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:27.871275 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f9a80-33d9-4fd0-8064-23f6f4f18139 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:27.871913 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889f9a80-33d9-4fd0-8064-23f6f4f18139 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:27.873594 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889f9a80-33d9-4fd0-8064-23f6f4f18139 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:27.884508 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1069/2142] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:27.889379 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6284108-acad-4fd0-b8c7-588a54e0eb0f admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:27.899770 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6284108-acad-4fd0-b8c7-588a54e0eb0f admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:27.904956 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb1372a-2940-4705-a290-a977f8d42979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:27.905614 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb1372a-2940-4705-a290-a977f8d42979 admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:27.905886 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb1372a-2940-4705-a290-a977f8d42979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:27.906162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb1372a-2940-4705-a290-a977f8d42979 admin admin] PATH_INFO: `/v3/projects/9375fb38ac7a40c1aba16d4185ae0dd5/users/7839cf561cda46e1a32100e697b71e09/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:27.920615 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb1372a-2940-4705-a290-a977f8d42979 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9375fb38ac7a40c1aba16d4185ae0dd5, user_id=7839cf561cda46e1a32100e697b71e09, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:27.923305 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb1372a-2940-4705-a290-a977f8d42979 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:27.937444 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1074/2143] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:27 2022] PUT /identity/v3/projects/9375fb38ac7a40c1aba16d4185ae0dd5/users/7839cf561cda46e1a32100e697b71e09/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:43:27.947833 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a66e1-4a10-4b87-8f74-0b194b9a16dd None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:27.948176 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a66e1-4a10-4b87-8f74-0b194b9a16dd None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:27.948446 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a66e1-4a10-4b87-8f74-0b194b9a16dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:27.977577 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-3d3a66e1-4a10-4b87-8f74-0b194b9a16dd None None] MFA Rules not processed for user `7839cf561cda46e1a32100e697b71e09`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:27.978698 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-3d3a66e1-4a10-4b87-8f74-0b194b9a16dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:28.044792 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1070/2144] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:28.055824 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb1372a-2940-4705-a290-a977f8d42979 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:28.057610 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-5fb1372a-2940-4705-a290-a977f8d42979 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:28.115646 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb1372a-2940-4705-a290-a977f8d42979 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:28.119355 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd9e312-c115-4b78-9481-953551feeda0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:28.119951 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd9e312-c115-4b78-9481-953551feeda0 service cinder] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:28.120229 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd9e312-c115-4b78-9481-953551feeda0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:28.120500 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd9e312-c115-4b78-9481-953551feeda0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:28.129457 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd9e312-c115-4b78-9481-953551feeda0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:28.130870 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd9e312-c115-4b78-9481-953551feeda0 service cinder] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:28.140296 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1075/2145] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:43:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:43:33.562942 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a66e1-4a10-4b87-8f74-0b194b9a16dd None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:33.564397 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-3d3a66e1-4a10-4b87-8f74-0b194b9a16dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:33.577154 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a66e1-4a10-4b87-8f74-0b194b9a16dd None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:33.580922 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54aba73-d10c-4366-84e2-8ad1b5249c99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:33.581518 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54aba73-d10c-4366-84e2-8ad1b5249c99 service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:33.581795 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54aba73-d10c-4366-84e2-8ad1b5249c99 service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:33.582066 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54aba73-d10c-4366-84e2-8ad1b5249c99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:33.583754 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-e54aba73-d10c-4366-84e2-8ad1b5249c99 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:33.646350 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54aba73-d10c-4366-84e2-8ad1b5249c99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:33.647806 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54aba73-d10c-4366-84e2-8ad1b5249c99 service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:33.657915 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1071/2146] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:43:33 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:43:33.976486 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd9e312-c115-4b78-9481-953551feeda0 service cinder] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:33.978359 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-5dd9e312-c115-4b78-9481-953551feeda0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:34.054809 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd9e312-c115-4b78-9481-953551feeda0 service cinder] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:34.059990 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bca594-c9cb-4bb8-9789-9980ed632945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:34.060648 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bca594-c9cb-4bb8-9789-9980ed632945 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:34.060938 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bca594-c9cb-4bb8-9789-9980ed632945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:34.061203 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bca594-c9cb-4bb8-9789-9980ed632945 admin admin] PATH_INFO: `/v3/users/bb134500cecb4fbab05a05a279a46a9b` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:34.062370 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bca594-c9cb-4bb8-9789-9980ed632945 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb134500cecb4fbab05a05a279a46a9b)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:34.064338 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bca594-c9cb-4bb8-9789-9980ed632945 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:34.115955 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-a3bca594-c9cb-4bb8-9789-9980ed632945 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb134500cecb4fbab05a05a279a46a9b'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:34.116669 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-a3bca594-c9cb-4bb8-9789-9980ed632945 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb134500cecb4fbab05a05a279a46a9b'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:34.124249 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-a3bca594-c9cb-4bb8-9789-9980ed632945 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb134500cecb4fbab05a05a279a46a9b'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:34.131510 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-a3bca594-c9cb-4bb8-9789-9980ed632945 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb134500cecb4fbab05a05a279a46a9b'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:34.132174 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-a3bca594-c9cb-4bb8-9789-9980ed632945 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb134500cecb4fbab05a05a279a46a9b'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:34.133329 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:INSERT=3,keystone:SELECT=89,keystone:DELETE=8 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:34.151008 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1076/2147] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:43:33 2022] DELETE /identity/v3/users/bb134500cecb4fbab05a05a279a46a9b => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:34.156164 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b4717-d4c7-4494-a92a-012f6b338f11 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:34.156448 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b4717-d4c7-4494-a92a-012f6b338f11 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:34.156719 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b4717-d4c7-4494-a92a-012f6b338f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:34.176136 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-298b4717-d4c7-4494-a92a-012f6b338f11 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:34.177329 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-298b4717-d4c7-4494-a92a-012f6b338f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:34.238158 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1072/2148] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:34.248023 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bca594-c9cb-4bb8-9789-9980ed632945 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:34.249403 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-a3bca594-c9cb-4bb8-9789-9980ed632945 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:34.305799 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bca594-c9cb-4bb8-9789-9980ed632945 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:34.309415 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb087a0-c3d3-491b-9648-aaec5a6b854b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:34.310206 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb087a0-c3d3-491b-9648-aaec5a6b854b service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:34.310480 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb087a0-c3d3-491b-9648-aaec5a6b854b service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:34.310747 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb087a0-c3d3-491b-9648-aaec5a6b854b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:34.319774 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb087a0-c3d3-491b-9648-aaec5a6b854b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:34.321175 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb087a0-c3d3-491b-9648-aaec5a6b854b service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:34.330432 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1077/2149] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:43:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:43:34.697669 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b4717-d4c7-4494-a92a-012f6b338f11 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:34.699756 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-298b4717-d4c7-4494-a92a-012f6b338f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:34.709459 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:34.716813 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b4717-d4c7-4494-a92a-012f6b338f11 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:34.723642 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73668c58-968f-45ec-8b59-5a5c50fc1623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:34.724327 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73668c58-968f-45ec-8b59-5a5c50fc1623 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:34.724570 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73668c58-968f-45ec-8b59-5a5c50fc1623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:34.724890 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73668c58-968f-45ec-8b59-5a5c50fc1623 admin admin] PATH_INFO: `/v3/projects/46715c44599c4522a88f8ca617447c9c` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:34.725962 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73668c58-968f-45ec-8b59-5a5c50fc1623 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46715c44599c4522a88f8ca617447c9c)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:34.727919 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73668c58-968f-45ec-8b59-5a5c50fc1623 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:34.776095 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-73668c58-968f-45ec-8b59-5a5c50fc1623 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46715c44599c4522a88f8ca617447c9c', 'request_id': 'req-73668c58-968f-45ec-8b59-5a5c50fc1623'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:34.781969 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-73668c58-968f-45ec-8b59-5a5c50fc1623 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46715c44599c4522a88f8ca617447c9c', 'request_id': 'req-73668c58-968f-45ec-8b59-5a5c50fc1623'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:34.788357 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-73668c58-968f-45ec-8b59-5a5c50fc1623 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46715c44599c4522a88f8ca617447c9c', 'request_id': 'req-73668c58-968f-45ec-8b59-5a5c50fc1623'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:34.802141 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-73668c58-968f-45ec-8b59-5a5c50fc1623 admin admin] The token cache is being invalidate because project 46715c44599c4522a88f8ca617447c9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:43:34.802436 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-73668c58-968f-45ec-8b59-5a5c50fc1623 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:34.803156 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-73668c58-968f-45ec-8b59-5a5c50fc1623 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:34.804799 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1073/2150] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:43:34 2022] DELETE /identity/v3/projects/46715c44599c4522a88f8ca617447c9c => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:34.808914 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb087a0-c3d3-491b-9648-aaec5a6b854b service neutron] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:34.819354 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb087a0-c3d3-491b-9648-aaec5a6b854b service neutron] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:34.875760 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec23f7-ba19-41b3-9e04-951c824f171c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:34.876383 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec23f7-ba19-41b3-9e04-951c824f171c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:34.876659 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec23f7-ba19-41b3-9e04-951c824f171c admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:34.876929 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec23f7-ba19-41b3-9e04-951c824f171c admin admin] PATH_INFO: `/v3/users/d31de227c57643839584038c5bb8873a` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:34.878048 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ec23f7-ba19-41b3-9e04-951c824f171c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d31de227c57643839584038c5bb8873a)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:34.880000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ec23f7-ba19-41b3-9e04-951c824f171c admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:34.932519 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-10ec23f7-ba19-41b3-9e04-951c824f171c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd31de227c57643839584038c5bb8873a'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:34.933421 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-10ec23f7-ba19-41b3-9e04-951c824f171c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd31de227c57643839584038c5bb8873a'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:34.941306 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-10ec23f7-ba19-41b3-9e04-951c824f171c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd31de227c57643839584038c5bb8873a'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:34.948632 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-10ec23f7-ba19-41b3-9e04-951c824f171c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd31de227c57643839584038c5bb8873a'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:34.949329 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-10ec23f7-ba19-41b3-9e04-951c824f171c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd31de227c57643839584038c5bb8873a'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:34.968064 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1078/2151] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:43:34 2022] DELETE /identity/v3/users/d31de227c57643839584038c5bb8873a => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:35.455405 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73668c58-968f-45ec-8b59-5a5c50fc1623 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:35.457163 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-73668c58-968f-45ec-8b59-5a5c50fc1623 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:35.477605 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73668c58-968f-45ec-8b59-5a5c50fc1623 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:35.535398 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9ee961-d8ff-4626-b8c0-13570b38cfef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:35.535964 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9ee961-d8ff-4626-b8c0-13570b38cfef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:35.536250 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9ee961-d8ff-4626-b8c0-13570b38cfef admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:35.536524 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9ee961-d8ff-4626-b8c0-13570b38cfef admin admin] PATH_INFO: `/v3/projects/3305e0e019484954ad9ae930b09e6849` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:35.537657 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9ee961-d8ff-4626-b8c0-13570b38cfef admin admin] RBAC: Authorizing `identity:delete_project(project_id=3305e0e019484954ad9ae930b09e6849)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:35.539608 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9ee961-d8ff-4626-b8c0-13570b38cfef admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:35.590418 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-7d9ee961-d8ff-4626-b8c0-13570b38cfef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3305e0e019484954ad9ae930b09e6849', 'request_id': 'req-7d9ee961-d8ff-4626-b8c0-13570b38cfef'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:35.597014 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-7d9ee961-d8ff-4626-b8c0-13570b38cfef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3305e0e019484954ad9ae930b09e6849', 'request_id': 'req-7d9ee961-d8ff-4626-b8c0-13570b38cfef'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:35.603361 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-7d9ee961-d8ff-4626-b8c0-13570b38cfef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3305e0e019484954ad9ae930b09e6849', 'request_id': 'req-7d9ee961-d8ff-4626-b8c0-13570b38cfef'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:35.617042 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-7d9ee961-d8ff-4626-b8c0-13570b38cfef admin admin] The token cache is being invalidate because project 3305e0e019484954ad9ae930b09e6849 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:43:35.617335 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-7d9ee961-d8ff-4626-b8c0-13570b38cfef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:35.617995 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-7d9ee961-d8ff-4626-b8c0-13570b38cfef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:35.619640 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1074/2152] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:43:35 2022] DELETE /identity/v3/projects/3305e0e019484954ad9ae930b09e6849 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:35.624884 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc579bf7-2287-4491-bb54-0c2dc5d181e5 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:35.625320 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc579bf7-2287-4491-bb54-0c2dc5d181e5 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:35.625464 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc579bf7-2287-4491-bb54-0c2dc5d181e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:35.633409 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:35.647149 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-bc579bf7-2287-4491-bb54-0c2dc5d181e5 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:35.648687 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-bc579bf7-2287-4491-bb54-0c2dc5d181e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:35.723051 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1079/2153] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:35.730528 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d02cb5-c0d5-48d9-815f-016a17129504 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:35.731050 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d02cb5-c0d5-48d9-815f-016a17129504 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:35.731127 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d02cb5-c0d5-48d9-815f-016a17129504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:35.759858 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-f2d02cb5-c0d5-48d9-815f-016a17129504 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:35.761353 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-f2d02cb5-c0d5-48d9-815f-016a17129504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:35.773515 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1075/2154] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:35.778031 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc579bf7-2287-4491-bb54-0c2dc5d181e5 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:35.790760 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc579bf7-2287-4491-bb54-0c2dc5d181e5 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:35.796324 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af8eaf-59e2-452d-b741-7ee533e2343c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:35.797043 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af8eaf-59e2-452d-b741-7ee533e2343c admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:35.797328 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af8eaf-59e2-452d-b741-7ee533e2343c admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:35.797626 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af8eaf-59e2-452d-b741-7ee533e2343c admin admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:35.798384 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06af8eaf-59e2-452d-b741-7ee533e2343c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:43:35.798748 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06af8eaf-59e2-452d-b741-7ee533e2343c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:35.800897 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06af8eaf-59e2-452d-b741-7ee533e2343c admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:35.815181 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1080/2155] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 14:43:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:43:35.820476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d02cb5-c0d5-48d9-815f-016a17129504 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:35.831830 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d02cb5-c0d5-48d9-815f-016a17129504 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:35.836776 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1255011-f00a-411c-96ca-1faf43d84045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:35.837476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1255011-f00a-411c-96ca-1faf43d84045 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:35.837762 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1255011-f00a-411c-96ca-1faf43d84045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:35.838182 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1255011-f00a-411c-96ca-1faf43d84045 admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:35.838829 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1255011-f00a-411c-96ca-1faf43d84045 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:35.841190 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1255011-f00a-411c-96ca-1faf43d84045 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:35.847328 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:35.886049 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1076/2156] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:43:35 2022] POST /identity/v3/projects => generated 387 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:35.890551 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af8eaf-59e2-452d-b741-7ee533e2343c admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:35.907107 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af8eaf-59e2-452d-b741-7ee533e2343c admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:36.002267 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4ef04e-140e-4a72-9143-7a41a367f5fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:36.002880 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ef04e-140e-4a72-9143-7a41a367f5fa admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:36.003153 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ef04e-140e-4a72-9143-7a41a367f5fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:36.003454 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ef04e-140e-4a72-9143-7a41a367f5fa admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:36.004100 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4ef04e-140e-4a72-9143-7a41a367f5fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:36.006039 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4ef04e-140e-4a72-9143-7a41a367f5fa admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:36.028903 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1081/2157] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:43:35 2022] POST /identity/v3/users => generated 294 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:36.033374 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1255011-f00a-411c-96ca-1faf43d84045 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:36.044390 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1255011-f00a-411c-96ca-1faf43d84045 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:36.049164 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3edb644-1e4a-438f-965c-dabc4d3a14cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:36.049765 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3edb644-1e4a-438f-965c-dabc4d3a14cd admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:36.050047 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3edb644-1e4a-438f-965c-dabc4d3a14cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:36.050321 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3edb644-1e4a-438f-965c-dabc4d3a14cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:36.050966 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3edb644-1e4a-438f-965c-dabc4d3a14cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:36.052609 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3edb644-1e4a-438f-965c-dabc4d3a14cd admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:36.064914 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1077/2158] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:36.069507 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4ef04e-140e-4a72-9143-7a41a367f5fa admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:36.080363 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4ef04e-140e-4a72-9143-7a41a367f5fa admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:36.085127 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d989770d-561c-45bc-bf40-5ea7a3808d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:36.085737 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d989770d-561c-45bc-bf40-5ea7a3808d61 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:36.086014 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d989770d-561c-45bc-bf40-5ea7a3808d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:36.086291 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d989770d-561c-45bc-bf40-5ea7a3808d61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:36.086963 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d989770d-561c-45bc-bf40-5ea7a3808d61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:36.088610 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d989770d-561c-45bc-bf40-5ea7a3808d61 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:36.100426 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1082/2159] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:43:36.104664 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3edb644-1e4a-438f-965c-dabc4d3a14cd admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:36.115868 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3edb644-1e4a-438f-965c-dabc4d3a14cd admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:36.120844 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad3d97-a215-4e1d-bb52-22b11b11469e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:36.121515 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad3d97-a215-4e1d-bb52-22b11b11469e admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:36.121796 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad3d97-a215-4e1d-bb52-22b11b11469e admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:36.122641 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad3d97-a215-4e1d-bb52-22b11b11469e admin admin] PATH_INFO: `/v3/projects/25c262acc52849a99cb01377569d9375/users/53a3d1351dae41c6b9b098bf0216642b/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:36.141700 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ad3d97-a215-4e1d-bb52-22b11b11469e admin admin] RBAC: Authorizing `identity:create_grant(project_id=25c262acc52849a99cb01377569d9375, user_id=53a3d1351dae41c6b9b098bf0216642b, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:36.144883 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ad3d97-a215-4e1d-bb52-22b11b11469e admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:36.165171 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1078/2160] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:36 2022] PUT /identity/v3/projects/25c262acc52849a99cb01377569d9375/users/53a3d1351dae41c6b9b098bf0216642b/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:36.173898 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e51eb-4b9f-449a-9173-4ff0ae71e547 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:36.174183 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e51eb-4b9f-449a-9173-4ff0ae71e547 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:36.174452 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e51eb-4b9f-449a-9173-4ff0ae71e547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:36.221582 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-b36e51eb-4b9f-449a-9173-4ff0ae71e547 None None] MFA Rules not processed for user `53a3d1351dae41c6b9b098bf0216642b`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:36.222844 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-b36e51eb-4b9f-449a-9173-4ff0ae71e547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:36.285604 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad3d97-a215-4e1d-bb52-22b11b11469e admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:36.287598 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-f7ad3d97-a215-4e1d-bb52-22b11b11469e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:36.300824 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:36.313085 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1083/2161] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:36 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:36.313571 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e51eb-4b9f-449a-9173-4ff0ae71e547 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:36.370749 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad3d97-a215-4e1d-bb52-22b11b11469e admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:36.378643 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dd6031-a956-40fb-bbe5-1c6b2f7a8683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:36.379274 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dd6031-a956-40fb-bbe5-1c6b2f7a8683 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:36.379274 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dd6031-a956-40fb-bbe5-1c6b2f7a8683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:36.379620 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dd6031-a956-40fb-bbe5-1c6b2f7a8683 admin admin] PATH_INFO: `/v3/users/b309b1abd7604db5acf80c4cfbf96982` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:36.381924 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dd6031-a956-40fb-bbe5-1c6b2f7a8683 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b309b1abd7604db5acf80c4cfbf96982)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:36.384897 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dd6031-a956-40fb-bbe5-1c6b2f7a8683 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:36.394366 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e51eb-4b9f-449a-9173-4ff0ae71e547 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:36.399135 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07636f08-edb5-478a-a092-4cd506ead60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:36.400948 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07636f08-edb5-478a-a092-4cd506ead60d admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:36.400948 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07636f08-edb5-478a-a092-4cd506ead60d admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:36.400948 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07636f08-edb5-478a-a092-4cd506ead60d admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:36.401577 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07636f08-edb5-478a-a092-4cd506ead60d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:36.404524 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07636f08-edb5-478a-a092-4cd506ead60d admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:36.459914 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1084/2162] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:43:36 2022] POST /identity/v3/projects => generated 387 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:36.471753 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07636f08-edb5-478a-a092-4cd506ead60d admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:36.480306 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-17dd6031-a956-40fb-bbe5-1c6b2f7a8683 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b309b1abd7604db5acf80c4cfbf96982'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:36.481246 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-17dd6031-a956-40fb-bbe5-1c6b2f7a8683 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b309b1abd7604db5acf80c4cfbf96982'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:36.502393 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-17dd6031-a956-40fb-bbe5-1c6b2f7a8683 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b309b1abd7604db5acf80c4cfbf96982'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:36.514994 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-17dd6031-a956-40fb-bbe5-1c6b2f7a8683 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b309b1abd7604db5acf80c4cfbf96982'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:36.527171 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-17dd6031-a956-40fb-bbe5-1c6b2f7a8683 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b309b1abd7604db5acf80c4cfbf96982'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:36.538476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1079/2163] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:43:36 2022] DELETE /identity/v3/users/b309b1abd7604db5acf80c4cfbf96982 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:36.542745 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d233b-3447-4c47-951b-192bb006dd1d None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:36.543351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d233b-3447-4c47-951b-192bb006dd1d None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:36.543770 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d233b-3447-4c47-951b-192bb006dd1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:36.589476 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07636f08-edb5-478a-a092-4cd506ead60d admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:36.596156 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-854d233b-3447-4c47-951b-192bb006dd1d None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:36.602415 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-854d233b-3447-4c47-951b-192bb006dd1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:36.658256 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:36.686707 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d084b0a7-990e-413a-ab52-75d99c326aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:36.687622 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084b0a7-990e-413a-ab52-75d99c326aec admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:36.687906 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084b0a7-990e-413a-ab52-75d99c326aec admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:36.688986 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084b0a7-990e-413a-ab52-75d99c326aec admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:36.689868 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d084b0a7-990e-413a-ab52-75d99c326aec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:36.693871 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d084b0a7-990e-413a-ab52-75d99c326aec admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:36.721846 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1085/2164] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:43:36 2022] POST /identity/v3/users => generated 300 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:36.722938 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d084b0a7-990e-413a-ab52-75d99c326aec admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:36.725130 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-d084b0a7-990e-413a-ab52-75d99c326aec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:36.733974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1080/2165] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:36.743050 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d084b0a7-990e-413a-ab52-75d99c326aec admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:36.745408 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854d233b-3447-4c47-951b-192bb006dd1d None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:36.746748 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-854d233b-3447-4c47-951b-192bb006dd1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:36.753553 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c537640a-14d9-431b-ba6f-e61afda69f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:36.754417 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537640a-14d9-431b-ba6f-e61afda69f6b admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:36.754863 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537640a-14d9-431b-ba6f-e61afda69f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:36.755318 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537640a-14d9-431b-ba6f-e61afda69f6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:36.756195 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c537640a-14d9-431b-ba6f-e61afda69f6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:36.758124 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c537640a-14d9-431b-ba6f-e61afda69f6b admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:36.769975 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1086/2166] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:36 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:36.774362 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c537640a-14d9-431b-ba6f-e61afda69f6b admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:36.786740 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c537640a-14d9-431b-ba6f-e61afda69f6b admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:36.793138 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d012c80-fa5d-494e-8bb6-39255cfd573e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:36.794660 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d012c80-fa5d-494e-8bb6-39255cfd573e admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:36.795181 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d012c80-fa5d-494e-8bb6-39255cfd573e admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:36.795623 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d012c80-fa5d-494e-8bb6-39255cfd573e admin admin] PATH_INFO: `/v3/projects/c03cbf65882d4dcc9a9e7be07642b14e/users/f7fd6b60df354853999fdf05d671ab67/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:36.815759 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:36.822287 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854d233b-3447-4c47-951b-192bb006dd1d None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:36.827659 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3419bbc8-42ac-4d9d-85d3-fb1339b5ef9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:36.828461 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419bbc8-42ac-4d9d-85d3-fb1339b5ef9a service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:36.828743 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419bbc8-42ac-4d9d-85d3-fb1339b5ef9a service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:36.829004 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419bbc8-42ac-4d9d-85d3-fb1339b5ef9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:36.835553 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d012c80-fa5d-494e-8bb6-39255cfd573e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c03cbf65882d4dcc9a9e7be07642b14e, user_id=f7fd6b60df354853999fdf05d671ab67, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:36.839523 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d012c80-fa5d-494e-8bb6-39255cfd573e admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:36.840944 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3419bbc8-42ac-4d9d-85d3-fb1339b5ef9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:36.842726 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3419bbc8-42ac-4d9d-85d3-fb1339b5ef9a service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:36.858172 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1087/2167] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:36 2022] PUT /identity/v3/projects/c03cbf65882d4dcc9a9e7be07642b14e/users/f7fd6b60df354853999fdf05d671ab67/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:43:36.859779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1081/2168] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:43:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:43:36.861929 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d012c80-fa5d-494e-8bb6-39255cfd573e admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:36.933868 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d012c80-fa5d-494e-8bb6-39255cfd573e admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:36.939903 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3cb528-c683-4af5-a13c-b114a10738d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:36.940899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3cb528-c683-4af5-a13c-b114a10738d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:36.941187 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3cb528-c683-4af5-a13c-b114a10738d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:36.942069 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3cb528-c683-4af5-a13c-b114a10738d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:36.942822 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3cb528-c683-4af5-a13c-b114a10738d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:36.944575 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3cb528-c683-4af5-a13c-b114a10738d5 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:36.961501 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1088/2169] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:36 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:43:36.962601 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3419bbc8-42ac-4d9d-85d3-fb1339b5ef9a service neutron] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:36.976753 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3419bbc8-42ac-4d9d-85d3-fb1339b5ef9a service neutron] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:37.003551 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee515e-dfe3-420a-b35e-35e87cffde26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:37.003551 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee515e-dfe3-420a-b35e-35e87cffde26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:37.003551 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee515e-dfe3-420a-b35e-35e87cffde26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:37.003841 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee515e-dfe3-420a-b35e-35e87cffde26 admin admin] PATH_INFO: `/v3/projects/c03cbf65882d4dcc9a9e7be07642b14e/users/f7fd6b60df354853999fdf05d671ab67/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:37.008821 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ee515e-dfe3-420a-b35e-35e87cffde26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c03cbf65882d4dcc9a9e7be07642b14e, user_id=f7fd6b60df354853999fdf05d671ab67, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:37.011506 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ee515e-dfe3-420a-b35e-35e87cffde26 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:37.026919 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1082/2170] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:36 2022] PUT /identity/v3/projects/c03cbf65882d4dcc9a9e7be07642b14e/users/f7fd6b60df354853999fdf05d671ab67/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:37.039340 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df997c3-435f-4b1c-9071-1ae81a84854e None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:37.039620 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df997c3-435f-4b1c-9071-1ae81a84854e None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:37.039959 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df997c3-435f-4b1c-9071-1ae81a84854e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:37.095571 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-7df997c3-435f-4b1c-9071-1ae81a84854e None None] MFA Rules not processed for user `f7fd6b60df354853999fdf05d671ab67`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:37.096950 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-7df997c3-435f-4b1c-9071-1ae81a84854e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:37.190161 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1089/2171] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:37 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:37.288855 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee515e-dfe3-420a-b35e-35e87cffde26 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:37.291196 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-d6ee515e-dfe3-420a-b35e-35e87cffde26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:37.326097 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df997c3-435f-4b1c-9071-1ae81a84854e None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:37.327880 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-7df997c3-435f-4b1c-9071-1ae81a84854e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:37.405526 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df997c3-435f-4b1c-9071-1ae81a84854e None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:37.412099 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa5860d-db51-4caf-91b8-89175b09a853 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:37.413111 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa5860d-db51-4caf-91b8-89175b09a853 service cinder] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:37.413379 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa5860d-db51-4caf-91b8-89175b09a853 service cinder] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:37.413725 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa5860d-db51-4caf-91b8-89175b09a853 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:37.435862 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa5860d-db51-4caf-91b8-89175b09a853 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:37.438246 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa5860d-db51-4caf-91b8-89175b09a853 service cinder] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:37.453845 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee515e-dfe3-420a-b35e-35e87cffde26 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:37.459735 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1090/2172] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:43:37 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:43:37.460742 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f608a52-7c0b-4dff-86d9-5cc4316c994f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:37.461590 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f608a52-7c0b-4dff-86d9-5cc4316c994f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:37.462061 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f608a52-7c0b-4dff-86d9-5cc4316c994f admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:37.462419 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f608a52-7c0b-4dff-86d9-5cc4316c994f admin admin] PATH_INFO: `/v3/projects/e1035b00467b43fcb44f7bbf04935650` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:37.463690 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f608a52-7c0b-4dff-86d9-5cc4316c994f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1035b00467b43fcb44f7bbf04935650)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:37.465603 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f608a52-7c0b-4dff-86d9-5cc4316c994f admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:37.519837 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-1f608a52-7c0b-4dff-86d9-5cc4316c994f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1035b00467b43fcb44f7bbf04935650', 'request_id': 'req-1f608a52-7c0b-4dff-86d9-5cc4316c994f'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:37.525873 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-1f608a52-7c0b-4dff-86d9-5cc4316c994f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1035b00467b43fcb44f7bbf04935650', 'request_id': 'req-1f608a52-7c0b-4dff-86d9-5cc4316c994f'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:37.534155 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-1f608a52-7c0b-4dff-86d9-5cc4316c994f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1035b00467b43fcb44f7bbf04935650', 'request_id': 'req-1f608a52-7c0b-4dff-86d9-5cc4316c994f'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:37.536043 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:37.553586 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-1f608a52-7c0b-4dff-86d9-5cc4316c994f admin admin] The token cache is being invalidate because project e1035b00467b43fcb44f7bbf04935650 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:43:37.554218 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-1f608a52-7c0b-4dff-86d9-5cc4316c994f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:37.554931 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-1f608a52-7c0b-4dff-86d9-5cc4316c994f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:37.562263 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1083/2173] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:43:37 2022] DELETE /identity/v3/projects/e1035b00467b43fcb44f7bbf04935650 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:37.563995 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f932293-37c9-44e7-823f-d3439cc846c8 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:37.564530 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f932293-37c9-44e7-823f-d3439cc846c8 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:37.564815 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f932293-37c9-44e7-823f-d3439cc846c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:37.587362 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-4f932293-37c9-44e7-823f-d3439cc846c8 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:37.587362 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-4f932293-37c9-44e7-823f-d3439cc846c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:37.592595 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:37.658814 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1091/2174] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:37.666749 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791cb82c-572f-45c5-99d3-c79aaac84f6f None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:37.667035 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791cb82c-572f-45c5-99d3-c79aaac84f6f None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:37.667340 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791cb82c-572f-45c5-99d3-c79aaac84f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:37.692116 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-791cb82c-572f-45c5-99d3-c79aaac84f6f None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:37.693330 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-791cb82c-572f-45c5-99d3-c79aaac84f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:37.706152 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1084/2175] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:37.710611 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f932293-37c9-44e7-823f-d3439cc846c8 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:37.742578 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f932293-37c9-44e7-823f-d3439cc846c8 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:37.751847 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bbb16b-0c77-4ee7-83b1-febd5fd9e792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:37.753683 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bbb16b-0c77-4ee7-83b1-febd5fd9e792 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:37.754037 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bbb16b-0c77-4ee7-83b1-febd5fd9e792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:37.754374 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bbb16b-0c77-4ee7-83b1-febd5fd9e792 admin admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:37.755070 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bbb16b-0c77-4ee7-83b1-febd5fd9e792 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:43:37.756663 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bbb16b-0c77-4ee7-83b1-febd5fd9e792 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:37.759524 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bbb16b-0c77-4ee7-83b1-febd5fd9e792 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:37.774978 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1092/2176] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 14:43:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:43:37.780458 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791cb82c-572f-45c5-99d3-c79aaac84f6f None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:37.796417 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791cb82c-572f-45c5-99d3-c79aaac84f6f None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:37.804212 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f06d76b-93c0-467d-94f9-b1bc0a5c4443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:37.804934 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06d76b-93c0-467d-94f9-b1bc0a5c4443 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:37.805315 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06d76b-93c0-467d-94f9-b1bc0a5c4443 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:37.806105 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06d76b-93c0-467d-94f9-b1bc0a5c4443 admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:37.806621 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f06d76b-93c0-467d-94f9-b1bc0a5c4443 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:37.810866 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f06d76b-93c0-467d-94f9-b1bc0a5c4443 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:37.852695 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1085/2177] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:43:37 2022] POST /identity/v3/projects => generated 377 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:37.858358 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bbb16b-0c77-4ee7-83b1-febd5fd9e792 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:37.869411 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bbb16b-0c77-4ee7-83b1-febd5fd9e792 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:37.927113 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bec446-0b2f-4d45-a1a5-b5a8ea84aac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:37.927753 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bec446-0b2f-4d45-a1a5-b5a8ea84aac7 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:37.928029 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bec446-0b2f-4d45-a1a5-b5a8ea84aac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:37.928298 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bec446-0b2f-4d45-a1a5-b5a8ea84aac7 admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:37.928927 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bec446-0b2f-4d45-a1a5-b5a8ea84aac7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:37.930827 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bec446-0b2f-4d45-a1a5-b5a8ea84aac7 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:37.953498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1093/2178] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:43:37 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:37.958124 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f06d76b-93c0-467d-94f9-b1bc0a5c4443 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:37.973687 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f06d76b-93c0-467d-94f9-b1bc0a5c4443 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:37.978573 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ffe493-fc04-4035-9db7-8cec2e9d457e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:37.979176 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ffe493-fc04-4035-9db7-8cec2e9d457e admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:37.979483 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ffe493-fc04-4035-9db7-8cec2e9d457e admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:37.979759 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ffe493-fc04-4035-9db7-8cec2e9d457e admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:37.980402 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ffe493-fc04-4035-9db7-8cec2e9d457e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:37.982739 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ffe493-fc04-4035-9db7-8cec2e9d457e admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:37.993385 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1086/2179] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:37.997844 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bec446-0b2f-4d45-a1a5-b5a8ea84aac7 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:38.008048 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bec446-0b2f-4d45-a1a5-b5a8ea84aac7 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:38.013203 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337ea2af-b99e-4469-842b-71c8c35c26c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:38.013672 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ea2af-b99e-4469-842b-71c8c35c26c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:38.014582 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ea2af-b99e-4469-842b-71c8c35c26c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:38.014862 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ea2af-b99e-4469-842b-71c8c35c26c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:38.015580 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337ea2af-b99e-4469-842b-71c8c35c26c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:38.017166 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337ea2af-b99e-4469-842b-71c8c35c26c8 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:38.028664 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1094/2180] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:38.033086 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ffe493-fc04-4035-9db7-8cec2e9d457e admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:38.043150 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ffe493-fc04-4035-9db7-8cec2e9d457e admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:38.049247 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08092ce-1780-475a-b54e-cedb8db93e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:38.050121 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08092ce-1780-475a-b54e-cedb8db93e5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:38.050476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08092ce-1780-475a-b54e-cedb8db93e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:38.050756 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08092ce-1780-475a-b54e-cedb8db93e5d admin admin] PATH_INFO: `/v3/projects/1b1a27d46efd46bea9140505a4aab0cf/users/a96cf3acb4d54113bb4706a21bdb8486/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:38.069531 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08092ce-1780-475a-b54e-cedb8db93e5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b1a27d46efd46bea9140505a4aab0cf, user_id=a96cf3acb4d54113bb4706a21bdb8486, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:38.072200 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08092ce-1780-475a-b54e-cedb8db93e5d admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:38.085648 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1087/2181] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:38 2022] PUT /identity/v3/projects/1b1a27d46efd46bea9140505a4aab0cf/users/a96cf3acb4d54113bb4706a21bdb8486/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:38.096659 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ecfae-b55a-457f-a254-24bb9492eb9d None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:38.096945 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ecfae-b55a-457f-a254-24bb9492eb9d None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:38.097229 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ecfae-b55a-457f-a254-24bb9492eb9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:38.126743 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-819ecfae-b55a-457f-a254-24bb9492eb9d None None] MFA Rules not processed for user `a96cf3acb4d54113bb4706a21bdb8486`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:38.127987 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-819ecfae-b55a-457f-a254-24bb9492eb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:38.161301 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08092ce-1780-475a-b54e-cedb8db93e5d admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:38.163447 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-e08092ce-1780-475a-b54e-cedb8db93e5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:38.203839 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1095/2182] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:38.208327 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819ecfae-b55a-457f-a254-24bb9492eb9d None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:38.213597 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:38.243050 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08092ce-1780-475a-b54e-cedb8db93e5d admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:38.252631 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6160ee-cf8a-4805-b738-6f6ca9cb7775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:38.253881 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6160ee-cf8a-4805-b738-6f6ca9cb7775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:38.253881 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6160ee-cf8a-4805-b738-6f6ca9cb7775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:38.254153 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6160ee-cf8a-4805-b738-6f6ca9cb7775 admin admin] PATH_INFO: `/v3/users/53a3d1351dae41c6b9b098bf0216642b` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:38.255353 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6160ee-cf8a-4805-b738-6f6ca9cb7775 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53a3d1351dae41c6b9b098bf0216642b)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:38.257271 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6160ee-cf8a-4805-b738-6f6ca9cb7775 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:38.277497 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819ecfae-b55a-457f-a254-24bb9492eb9d None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:38.285788 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b6fab-8635-4e8b-b876-7ae1b087a9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:38.286467 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b6fab-8635-4e8b-b876-7ae1b087a9d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:38.286759 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b6fab-8635-4e8b-b876-7ae1b087a9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:38.287063 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b6fab-8635-4e8b-b876-7ae1b087a9d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:38.287805 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b6fab-8635-4e8b-b876-7ae1b087a9d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:38.289751 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b6fab-8635-4e8b-b876-7ae1b087a9d8 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:38.313241 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-9c6160ee-cf8a-4805-b738-6f6ca9cb7775 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53a3d1351dae41c6b9b098bf0216642b'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:38.314198 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-9c6160ee-cf8a-4805-b738-6f6ca9cb7775 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53a3d1351dae41c6b9b098bf0216642b'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:38.320443 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1096/2183] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:43:38 2022] POST /identity/v3/projects => generated 377 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:43:38.326117 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b6fab-8635-4e8b-b876-7ae1b087a9d8 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:38.332152 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-9c6160ee-cf8a-4805-b738-6f6ca9cb7775 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53a3d1351dae41c6b9b098bf0216642b'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:38.340462 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-9c6160ee-cf8a-4805-b738-6f6ca9cb7775 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53a3d1351dae41c6b9b098bf0216642b'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:38.349922 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-9c6160ee-cf8a-4805-b738-6f6ca9cb7775 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53a3d1351dae41c6b9b098bf0216642b'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:38.356970 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1088/2184] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:43:38 2022] DELETE /identity/v3/users/53a3d1351dae41c6b9b098bf0216642b => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:43:38.361312 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e65c5-6361-4229-82c5-69c4435f06fe None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:38.361664 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e65c5-6361-4229-82c5-69c4435f06fe None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:38.361946 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e65c5-6361-4229-82c5-69c4435f06fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:38.376328 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:INSERT=4,keystone:DELETE=11,keystone:SELECT=85 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:38.384082 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-cd7e65c5-6361-4229-82c5-69c4435f06fe None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:38.386065 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-cd7e65c5-6361-4229-82c5-69c4435f06fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:38.406007 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b6fab-8635-4e8b-b876-7ae1b087a9d8 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:38.459260 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1089/2185] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:38.470321 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7e65c5-6361-4229-82c5-69c4435f06fe None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:38.472425 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-cd7e65c5-6361-4229-82c5-69c4435f06fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:38.480313 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832982b6-76d7-42ad-9d6f-8173e6893cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:38.480993 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832982b6-76d7-42ad-9d6f-8173e6893cf6 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:38.481457 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832982b6-76d7-42ad-9d6f-8173e6893cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:38.482397 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832982b6-76d7-42ad-9d6f-8173e6893cf6 admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:38.483133 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832982b6-76d7-42ad-9d6f-8173e6893cf6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:38.485094 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832982b6-76d7-42ad-9d6f-8173e6893cf6 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:38.508293 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1097/2186] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:43:38 2022] POST /identity/v3/users => generated 289 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:43:38.512369 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832982b6-76d7-42ad-9d6f-8173e6893cf6 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:38.515042 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-832982b6-76d7-42ad-9d6f-8173e6893cf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:38.526814 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832982b6-76d7-42ad-9d6f-8173e6893cf6 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:38.532117 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558afc9a-ad0d-4d06-9e99-349e4731c94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:38.532702 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558afc9a-ad0d-4d06-9e99-349e4731c94c admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:38.532980 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558afc9a-ad0d-4d06-9e99-349e4731c94c admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:38.533258 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558afc9a-ad0d-4d06-9e99-349e4731c94c admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:38.533963 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558afc9a-ad0d-4d06-9e99-349e4731c94c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:38.535595 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558afc9a-ad0d-4d06-9e99-349e4731c94c admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:38.545479 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7e65c5-6361-4229-82c5-69c4435f06fe None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:38.545968 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1098/2187] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:43:38.549996 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558afc9a-ad0d-4d06-9e99-349e4731c94c admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:38.550767 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75076f23-3cd7-4055-9635-63d3eabd982e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:38.551435 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75076f23-3cd7-4055-9635-63d3eabd982e service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:38.551713 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75076f23-3cd7-4055-9635-63d3eabd982e service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:38.551986 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75076f23-3cd7-4055-9635-63d3eabd982e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:38.560664 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558afc9a-ad0d-4d06-9e99-349e4731c94c admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:38.563380 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75076f23-3cd7-4055-9635-63d3eabd982e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:38.564880 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75076f23-3cd7-4055-9635-63d3eabd982e service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:38.565881 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c37995-e0cf-4d41-ad3b-c6dede8d2be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:38.566508 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c37995-e0cf-4d41-ad3b-c6dede8d2be1 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:38.566792 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c37995-e0cf-4d41-ad3b-c6dede8d2be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:38.567068 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c37995-e0cf-4d41-ad3b-c6dede8d2be1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:38.567741 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c37995-e0cf-4d41-ad3b-c6dede8d2be1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:38.569351 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c37995-e0cf-4d41-ad3b-c6dede8d2be1 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:38.574906 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1090/2188] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:43:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:43:38.580997 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1099/2189] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:43:38.585272 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75076f23-3cd7-4055-9635-63d3eabd982e service neutron] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:38.595326 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75076f23-3cd7-4055-9635-63d3eabd982e service neutron] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:38.600065 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb52e45f-4ba6-40ef-9937-d80ba692f842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:38.600751 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52e45f-4ba6-40ef-9937-d80ba692f842 admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:38.601081 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52e45f-4ba6-40ef-9937-d80ba692f842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:38.601355 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52e45f-4ba6-40ef-9937-d80ba692f842 admin admin] PATH_INFO: `/v3/projects/38c645caf2e14858b9a117c04d1c7419/users/2224ce98ffa242d79e7fcb8e183be47a/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:38.616966 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb52e45f-4ba6-40ef-9937-d80ba692f842 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38c645caf2e14858b9a117c04d1c7419, user_id=2224ce98ffa242d79e7fcb8e183be47a, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:38.620202 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb52e45f-4ba6-40ef-9937-d80ba692f842 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:38.635255 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1091/2190] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:38 2022] PUT /identity/v3/projects/38c645caf2e14858b9a117c04d1c7419/users/2224ce98ffa242d79e7fcb8e183be47a/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:43:38.647226 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee6657-0e4d-4aeb-bab1-253f74f3dda1 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:38.647527 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee6657-0e4d-4aeb-bab1-253f74f3dda1 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:38.648003 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee6657-0e4d-4aeb-bab1-253f74f3dda1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:38.656599 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:38.677545 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-8bee6657-0e4d-4aeb-bab1-253f74f3dda1 None None] MFA Rules not processed for user `2224ce98ffa242d79e7fcb8e183be47a`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:38.678862 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-8bee6657-0e4d-4aeb-bab1-253f74f3dda1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:38.747482 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1100/2191] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:43:38.753025 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb52e45f-4ba6-40ef-9937-d80ba692f842 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:38.818878 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb52e45f-4ba6-40ef-9937-d80ba692f842 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:38.823580 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea1926f-0588-4bd2-a58c-3ef2ceea9136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:38.824188 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1926f-0588-4bd2-a58c-3ef2ceea9136 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:38.824466 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1926f-0588-4bd2-a58c-3ef2ceea9136 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:38.824740 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1926f-0588-4bd2-a58c-3ef2ceea9136 admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:38.826270 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea1926f-0588-4bd2-a58c-3ef2ceea9136 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:38.828182 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea1926f-0588-4bd2-a58c-3ef2ceea9136 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:38.860293 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1092/2192] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:43:38 2022] POST /identity/v3/projects => generated 375 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:38.864415 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bee6657-0e4d-4aeb-bab1-253f74f3dda1 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:38.867249 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea1926f-0588-4bd2-a58c-3ef2ceea9136 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:38.869160 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-3ea1926f-0588-4bd2-a58c-3ef2ceea9136 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:38.880368 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bee6657-0e4d-4aeb-bab1-253f74f3dda1 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:38.923201 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:38.956017 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aefd5c-bac1-457b-9370-59d3b34b504c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:38.956643 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aefd5c-bac1-457b-9370-59d3b34b504c admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:38.956928 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aefd5c-bac1-457b-9370-59d3b34b504c admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:38.957209 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aefd5c-bac1-457b-9370-59d3b34b504c admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:38.957966 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aefd5c-bac1-457b-9370-59d3b34b504c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:38.960058 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aefd5c-bac1-457b-9370-59d3b34b504c admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:38.982727 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea1926f-0588-4bd2-a58c-3ef2ceea9136 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:38.984603 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1101/2193] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:43:38 2022] POST /identity/v3/users => generated 294 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:38.988653 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aefd5c-bac1-457b-9370-59d3b34b504c admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:38.990549 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b176f130-7526-48d5-afff-f393679c7063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:38.991183 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b176f130-7526-48d5-afff-f393679c7063 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:38.991520 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b176f130-7526-48d5-afff-f393679c7063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:38.991793 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b176f130-7526-48d5-afff-f393679c7063 admin admin] PATH_INFO: `/v3/projects/25c262acc52849a99cb01377569d9375` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:38.992969 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b176f130-7526-48d5-afff-f393679c7063 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25c262acc52849a99cb01377569d9375)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:38.994886 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b176f130-7526-48d5-afff-f393679c7063 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:38.999514 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aefd5c-bac1-457b-9370-59d3b34b504c admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:39.004186 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d14b582-3047-4cb1-9ab8-499c4f3c88b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:39.004781 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d14b582-3047-4cb1-9ab8-499c4f3c88b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:39.005057 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d14b582-3047-4cb1-9ab8-499c4f3c88b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:39.005349 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d14b582-3047-4cb1-9ab8-499c4f3c88b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:39.005985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d14b582-3047-4cb1-9ab8-499c4f3c88b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:39.007605 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d14b582-3047-4cb1-9ab8-499c4f3c88b5 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:39.018996 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1102/2194] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:43:39.021710 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d14b582-3047-4cb1-9ab8-499c4f3c88b5 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:39.032606 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d14b582-3047-4cb1-9ab8-499c4f3c88b5 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:39.045589 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b176f130-7526-48d5-afff-f393679c7063 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25c262acc52849a99cb01377569d9375', 'request_id': 'req-b176f130-7526-48d5-afff-f393679c7063'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:39.051257 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b176f130-7526-48d5-afff-f393679c7063 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25c262acc52849a99cb01377569d9375', 'request_id': 'req-b176f130-7526-48d5-afff-f393679c7063'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:39.057857 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b176f130-7526-48d5-afff-f393679c7063 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25c262acc52849a99cb01377569d9375', 'request_id': 'req-b176f130-7526-48d5-afff-f393679c7063'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:39.073022 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b176f130-7526-48d5-afff-f393679c7063 admin admin] The token cache is being invalidate because project 25c262acc52849a99cb01377569d9375 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:43:39.073393 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b176f130-7526-48d5-afff-f393679c7063 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:39.074097 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b176f130-7526-48d5-afff-f393679c7063 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:39.075816 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1093/2195] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:43:38 2022] DELETE /identity/v3/projects/25c262acc52849a99cb01377569d9375 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:39.079738 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b176f130-7526-48d5-afff-f393679c7063 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:39.081015 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-b176f130-7526-48d5-afff-f393679c7063 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:39.094946 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41782eac-dce9-4985-941f-cb3c530af265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:39.095629 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41782eac-dce9-4985-941f-cb3c530af265 admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:39.096040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41782eac-dce9-4985-941f-cb3c530af265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:39.096314 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41782eac-dce9-4985-941f-cb3c530af265 admin admin] PATH_INFO: `/v3/projects/c86908735f7343ce90302e36ba62b9e5/users/971e94bab5e74fbca9786ebba4f09d61/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:39.111643 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41782eac-dce9-4985-941f-cb3c530af265 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c86908735f7343ce90302e36ba62b9e5, user_id=971e94bab5e74fbca9786ebba4f09d61, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:39.114283 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41782eac-dce9-4985-941f-cb3c530af265 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:39.117269 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:39.128507 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1103/2196] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:39 2022] PUT /identity/v3/projects/c86908735f7343ce90302e36ba62b9e5/users/971e94bab5e74fbca9786ebba4f09d61/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:43:39.132561 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41782eac-dce9-4985-941f-cb3c530af265 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:39.134177 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-41782eac-dce9-4985-941f-cb3c530af265 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:39.147928 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b176f130-7526-48d5-afff-f393679c7063 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:39.192178 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:39.200955 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41782eac-dce9-4985-941f-cb3c530af265 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:39.205789 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b78a837-06ed-4fba-b17e-7981067c5019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:39.206401 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b78a837-06ed-4fba-b17e-7981067c5019 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:39.206683 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b78a837-06ed-4fba-b17e-7981067c5019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:39.206850 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d85e3ee-3f0d-4181-b486-ff7c4df76ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:39.206957 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b78a837-06ed-4fba-b17e-7981067c5019 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:39.207648 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b78a837-06ed-4fba-b17e-7981067c5019 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:39.207766 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d85e3ee-3f0d-4181-b486-ff7c4df76ceb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:39.208246 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d85e3ee-3f0d-4181-b486-ff7c4df76ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:39.208623 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d85e3ee-3f0d-4181-b486-ff7c4df76ceb admin admin] PATH_INFO: `/v3/users/f7fd6b60df354853999fdf05d671ab67` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:39.209309 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b78a837-06ed-4fba-b17e-7981067c5019 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:39.210208 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d85e3ee-3f0d-4181-b486-ff7c4df76ceb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7fd6b60df354853999fdf05d671ab67)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:39.212478 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d85e3ee-3f0d-4181-b486-ff7c4df76ceb admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:39.220530 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1104/2197] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:39 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:43:39.224717 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b78a837-06ed-4fba-b17e-7981067c5019 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:39.235221 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b78a837-06ed-4fba-b17e-7981067c5019 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:39.239981 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40910131-d265-4459-ba84-25165d830184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:39.240688 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40910131-d265-4459-ba84-25165d830184 admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:39.241003 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40910131-d265-4459-ba84-25165d830184 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:39.241340 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40910131-d265-4459-ba84-25165d830184 admin admin] PATH_INFO: `/v3/projects/c86908735f7343ce90302e36ba62b9e5/users/971e94bab5e74fbca9786ebba4f09d61/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:39.243919 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40910131-d265-4459-ba84-25165d830184 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c86908735f7343ce90302e36ba62b9e5, user_id=971e94bab5e74fbca9786ebba4f09d61, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:39.247751 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40910131-d265-4459-ba84-25165d830184 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:39.258780 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1105/2198] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:39 2022] PUT /identity/v3/projects/c86908735f7343ce90302e36ba62b9e5/users/971e94bab5e74fbca9786ebba4f09d61/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:39.264790 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-0d85e3ee-3f0d-4181-b486-ff7c4df76ceb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7fd6b60df354853999fdf05d671ab67'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:39.265460 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-0d85e3ee-3f0d-4181-b486-ff7c4df76ceb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7fd6b60df354853999fdf05d671ab67'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:39.269406 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94914a84-b8fb-458b-9894-b638875331d9 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:39.269691 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94914a84-b8fb-458b-9894-b638875331d9 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:39.269973 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94914a84-b8fb-458b-9894-b638875331d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:39.277856 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-0d85e3ee-3f0d-4181-b486-ff7c4df76ceb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7fd6b60df354853999fdf05d671ab67'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:39.285175 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-0d85e3ee-3f0d-4181-b486-ff7c4df76ceb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7fd6b60df354853999fdf05d671ab67'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:39.293556 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-0d85e3ee-3f0d-4181-b486-ff7c4df76ceb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7fd6b60df354853999fdf05d671ab67'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:39.300949 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1094/2199] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:43:39 2022] DELETE /identity/v3/users/f7fd6b60df354853999fdf05d671ab67 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:43:39.301635 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-94914a84-b8fb-458b-9894-b638875331d9 None None] MFA Rules not processed for user `971e94bab5e74fbca9786ebba4f09d61`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:39.302876 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-94914a84-b8fb-458b-9894-b638875331d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:39.381393 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1106/2200] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:39 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:43:39.393737 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d85e3ee-3f0d-4181-b486-ff7c4df76ceb admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:39.395319 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-0d85e3ee-3f0d-4181-b486-ff7c4df76ceb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:39.465048 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d85e3ee-3f0d-4181-b486-ff7c4df76ceb admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:39.470815 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2188e168-fbf8-449c-b3f1-3525c8748e06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:39.471847 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188e168-fbf8-449c-b3f1-3525c8748e06 service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:39.472321 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188e168-fbf8-449c-b3f1-3525c8748e06 service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:39.474351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2188e168-fbf8-449c-b3f1-3525c8748e06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:39.476246 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-2188e168-fbf8-449c-b3f1-3525c8748e06 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:39.557890 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2188e168-fbf8-449c-b3f1-3525c8748e06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:39.559431 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2188e168-fbf8-449c-b3f1-3525c8748e06 service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:39.569759 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1095/2201] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:43:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:43:39.611817 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94914a84-b8fb-458b-9894-b638875331d9 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:39.613361 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-94914a84-b8fb-458b-9894-b638875331d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:39.700639 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94914a84-b8fb-458b-9894-b638875331d9 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:39.705966 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e740b58-f5fd-4a63-97a9-cdffd92d9b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:39.706662 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e740b58-f5fd-4a63-97a9-cdffd92d9b61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:39.707010 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e740b58-f5fd-4a63-97a9-cdffd92d9b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:39.707906 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e740b58-f5fd-4a63-97a9-cdffd92d9b61 admin admin] PATH_INFO: `/v3/projects/c03cbf65882d4dcc9a9e7be07642b14e` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:39.708642 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e740b58-f5fd-4a63-97a9-cdffd92d9b61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c03cbf65882d4dcc9a9e7be07642b14e)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:39.710999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e740b58-f5fd-4a63-97a9-cdffd92d9b61 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:39.727443 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:39.774938 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-1e740b58-f5fd-4a63-97a9-cdffd92d9b61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c03cbf65882d4dcc9a9e7be07642b14e', 'request_id': 'req-1e740b58-f5fd-4a63-97a9-cdffd92d9b61'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:39.781579 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-1e740b58-f5fd-4a63-97a9-cdffd92d9b61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c03cbf65882d4dcc9a9e7be07642b14e', 'request_id': 'req-1e740b58-f5fd-4a63-97a9-cdffd92d9b61'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:39.788015 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-1e740b58-f5fd-4a63-97a9-cdffd92d9b61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c03cbf65882d4dcc9a9e7be07642b14e', 'request_id': 'req-1e740b58-f5fd-4a63-97a9-cdffd92d9b61'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:39.802115 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-1e740b58-f5fd-4a63-97a9-cdffd92d9b61 admin admin] The token cache is being invalidate because project c03cbf65882d4dcc9a9e7be07642b14e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:43:39.802572 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-1e740b58-f5fd-4a63-97a9-cdffd92d9b61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:39.803280 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-1e740b58-f5fd-4a63-97a9-cdffd92d9b61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:39.804948 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1107/2202] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:43:39 2022] DELETE /identity/v3/projects/c03cbf65882d4dcc9a9e7be07642b14e => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:39.810764 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2bee4-3a72-4cc4-b504-408c063c56da None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:39.811046 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2bee4-3a72-4cc4-b504-408c063c56da None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:39.811348 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2bee4-3a72-4cc4-b504-408c063c56da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:39.849723 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-56a2bee4-3a72-4cc4-b504-408c063c56da None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:39.850941 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-56a2bee4-3a72-4cc4-b504-408c063c56da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:39.870149 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:39.930622 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1096/2203] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:39.939054 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c9472-9dc6-4f21-bc1a-65c04a27de27 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:39.939593 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c9472-9dc6-4f21-bc1a-65c04a27de27 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:39.939949 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c9472-9dc6-4f21-bc1a-65c04a27de27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:39.968066 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-a27c9472-9dc6-4f21-bc1a-65c04a27de27 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:39.969563 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-a27c9472-9dc6-4f21-bc1a-65c04a27de27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:39.977781 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1108/2204] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:39.982037 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a2bee4-3a72-4cc4-b504-408c063c56da None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:39.991727 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a2bee4-3a72-4cc4-b504-408c063c56da None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:39.996408 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3583f5ba-c535-4388-bd10-a63115f9466e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:39.997583 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583f5ba-c535-4388-bd10-a63115f9466e admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:39.997861 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583f5ba-c535-4388-bd10-a63115f9466e admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:39.998219 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583f5ba-c535-4388-bd10-a63115f9466e admin admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:39.998880 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3583f5ba-c535-4388-bd10-a63115f9466e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:43:39.999187 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3583f5ba-c535-4388-bd10-a63115f9466e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:40.000819 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3583f5ba-c535-4388-bd10-a63115f9466e admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:40.013882 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1097/2205] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 14:43:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:43:40.018233 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27c9472-9dc6-4f21-bc1a-65c04a27de27 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:40.027615 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27c9472-9dc6-4f21-bc1a-65c04a27de27 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:40.032463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea428994-592f-4e10-8c73-0399a233b8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:40.033073 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea428994-592f-4e10-8c73-0399a233b8a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:40.033353 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea428994-592f-4e10-8c73-0399a233b8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:40.033627 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea428994-592f-4e10-8c73-0399a233b8a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:40.034258 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea428994-592f-4e10-8c73-0399a233b8a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:40.036211 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea428994-592f-4e10-8c73-0399a233b8a0 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:40.065315 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1109/2206] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:43:40 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:40.069802 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3583f5ba-c535-4388-bd10-a63115f9466e admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:40.079794 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3583f5ba-c535-4388-bd10-a63115f9466e admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:40.134399 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dc75ec-f388-4f9b-81d8-76cb036106c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:40.134993 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc75ec-f388-4f9b-81d8-76cb036106c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:40.135301 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc75ec-f388-4f9b-81d8-76cb036106c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:40.135572 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc75ec-f388-4f9b-81d8-76cb036106c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:40.136255 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dc75ec-f388-4f9b-81d8-76cb036106c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:40.138175 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dc75ec-f388-4f9b-81d8-76cb036106c0 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:40.158794 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1098/2207] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:43:40 2022] POST /identity/v3/users => generated 291 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:40.162951 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea428994-592f-4e10-8c73-0399a233b8a0 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:40.173814 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea428994-592f-4e10-8c73-0399a233b8a0 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:40.178428 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99924ce-566d-42d5-b01a-5a6aab6080c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:40.179024 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99924ce-566d-42d5-b01a-5a6aab6080c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:40.179341 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99924ce-566d-42d5-b01a-5a6aab6080c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:40.179614 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99924ce-566d-42d5-b01a-5a6aab6080c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:40.180252 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99924ce-566d-42d5-b01a-5a6aab6080c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:40.181834 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99924ce-566d-42d5-b01a-5a6aab6080c5 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:40.191922 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1110/2208] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:40.195686 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dc75ec-f388-4f9b-81d8-76cb036106c0 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:40.205628 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dc75ec-f388-4f9b-81d8-76cb036106c0 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:40.210318 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88b7876-f43d-4376-bc60-cf15d06be41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:40.210921 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b7876-f43d-4376-bc60-cf15d06be41d admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:40.211226 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b7876-f43d-4376-bc60-cf15d06be41d admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:40.211509 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b7876-f43d-4376-bc60-cf15d06be41d admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:40.212182 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88b7876-f43d-4376-bc60-cf15d06be41d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:40.213766 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88b7876-f43d-4376-bc60-cf15d06be41d admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:40.225105 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1099/2209] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:40.229192 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99924ce-566d-42d5-b01a-5a6aab6080c5 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:40.239756 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99924ce-566d-42d5-b01a-5a6aab6080c5 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:40.244421 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7f7025-6257-443c-8344-a00a3fc69973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:40.245066 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f7025-6257-443c-8344-a00a3fc69973 admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:40.245347 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f7025-6257-443c-8344-a00a3fc69973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:40.245614 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f7025-6257-443c-8344-a00a3fc69973 admin admin] PATH_INFO: `/v3/projects/46b7868cbff043f3aa9072f795a0314b/users/4ef76352db6645299c9e9488063af91c/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:40.260814 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7f7025-6257-443c-8344-a00a3fc69973 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46b7868cbff043f3aa9072f795a0314b, user_id=4ef76352db6645299c9e9488063af91c, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:40.264127 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7f7025-6257-443c-8344-a00a3fc69973 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:40.276940 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1111/2210] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:40 2022] PUT /identity/v3/projects/46b7868cbff043f3aa9072f795a0314b/users/4ef76352db6645299c9e9488063af91c/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:40.288414 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de51ce-8697-4ba8-92af-dc59bc683391 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:40.288699 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de51ce-8697-4ba8-92af-dc59bc683391 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:40.288972 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de51ce-8697-4ba8-92af-dc59bc683391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:40.317611 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-42de51ce-8697-4ba8-92af-dc59bc683391 None None] MFA Rules not processed for user `4ef76352db6645299c9e9488063af91c`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:40.318810 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-42de51ce-8697-4ba8-92af-dc59bc683391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:40.382860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1100/2211] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:40 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:43:40.387414 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7f7025-6257-443c-8344-a00a3fc69973 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:40.450251 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7f7025-6257-443c-8344-a00a3fc69973 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:40.455149 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5d10d4-60f2-4228-854f-70ef178035dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:40.455801 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5d10d4-60f2-4228-854f-70ef178035dd admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:40.456082 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5d10d4-60f2-4228-854f-70ef178035dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:40.456354 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5d10d4-60f2-4228-854f-70ef178035dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:40.456997 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5d10d4-60f2-4228-854f-70ef178035dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:40.458887 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5d10d4-60f2-4228-854f-70ef178035dd admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:40.485055 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1112/2212] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:43:40 2022] POST /identity/v3/projects => generated 383 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:40.489109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42de51ce-8697-4ba8-92af-dc59bc683391 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:40.500505 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42de51ce-8697-4ba8-92af-dc59bc683391 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:40.512697 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:40.556631 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67198129-47d5-4848-84f8-600bce30cb16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:40.557230 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67198129-47d5-4848-84f8-600bce30cb16 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:40.557509 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67198129-47d5-4848-84f8-600bce30cb16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:40.557782 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67198129-47d5-4848-84f8-600bce30cb16 admin admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:40.558420 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67198129-47d5-4848-84f8-600bce30cb16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:40.560913 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67198129-47d5-4848-84f8-600bce30cb16 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:40.583150 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1101/2213] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:43:40 2022] POST /identity/v3/users => generated 298 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:40.587224 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5d10d4-60f2-4228-854f-70ef178035dd admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:40.598036 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5d10d4-60f2-4228-854f-70ef178035dd admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:40.602660 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf38b373-24ea-4253-a63a-331593d73754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:40.603301 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38b373-24ea-4253-a63a-331593d73754 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:40.603597 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38b373-24ea-4253-a63a-331593d73754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:40.603866 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38b373-24ea-4253-a63a-331593d73754 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:40.604500 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf38b373-24ea-4253-a63a-331593d73754 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:40.606066 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf38b373-24ea-4253-a63a-331593d73754 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:40.618117 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1113/2214] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:40.622732 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67198129-47d5-4848-84f8-600bce30cb16 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:40.632542 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67198129-47d5-4848-84f8-600bce30cb16 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:40.637487 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef53905-29da-44c2-b49d-b5b53ab7472b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:40.638135 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef53905-29da-44c2-b49d-b5b53ab7472b admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:40.638433 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef53905-29da-44c2-b49d-b5b53ab7472b admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:40.638758 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef53905-29da-44c2-b49d-b5b53ab7472b admin admin] PATH_INFO: `/v3/projects/965739d6967747baaae58db1381688f6/users/9f820d54ab0847fcb7851704f86f88f2/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:40.653683 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef53905-29da-44c2-b49d-b5b53ab7472b admin admin] RBAC: Authorizing `identity:create_grant(project_id=965739d6967747baaae58db1381688f6, user_id=9f820d54ab0847fcb7851704f86f88f2, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:40.656366 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef53905-29da-44c2-b49d-b5b53ab7472b admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:40.670263 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1102/2215] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:40 2022] PUT /identity/v3/projects/965739d6967747baaae58db1381688f6/users/9f820d54ab0847fcb7851704f86f88f2/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:40.674408 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf38b373-24ea-4253-a63a-331593d73754 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:40.681405 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:40.685241 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf38b373-24ea-4253-a63a-331593d73754 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:40.748807 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3604fd6b-fcba-4760-8be1-36a85dd17d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:40.749421 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604fd6b-fcba-4760-8be1-36a85dd17d67 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:40.749698 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604fd6b-fcba-4760-8be1-36a85dd17d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:40.749973 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604fd6b-fcba-4760-8be1-36a85dd17d67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:40.750620 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3604fd6b-fcba-4760-8be1-36a85dd17d67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:40.752261 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3604fd6b-fcba-4760-8be1-36a85dd17d67 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:40.762651 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1114/2216] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:40 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:40.767737 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef53905-29da-44c2-b49d-b5b53ab7472b admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:40.777712 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef53905-29da-44c2-b49d-b5b53ab7472b admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:40.782984 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d9043-6488-4e6e-8d3b-e3c8c6f6ed37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:40.783772 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d9043-6488-4e6e-8d3b-e3c8c6f6ed37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:40.784170 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d9043-6488-4e6e-8d3b-e3c8c6f6ed37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:40.784515 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d9043-6488-4e6e-8d3b-e3c8c6f6ed37 admin admin] PATH_INFO: `/v3/projects/965739d6967747baaae58db1381688f6/users/9f820d54ab0847fcb7851704f86f88f2/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:40.786604 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2d9043-6488-4e6e-8d3b-e3c8c6f6ed37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=965739d6967747baaae58db1381688f6, user_id=9f820d54ab0847fcb7851704f86f88f2, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:40.789327 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2d9043-6488-4e6e-8d3b-e3c8c6f6ed37 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:40.811493 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1103/2217] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:40 2022] PUT /identity/v3/projects/965739d6967747baaae58db1381688f6/users/9f820d54ab0847fcb7851704f86f88f2/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:40.813547 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9368fad-f310-49b5-9edd-d5e76d2c7481 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:40.814024 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9368fad-f310-49b5-9edd-d5e76d2c7481 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:40.814149 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9368fad-f310-49b5-9edd-d5e76d2c7481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:40.846101 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-b9368fad-f310-49b5-9edd-d5e76d2c7481 None None] MFA Rules not processed for user `9f820d54ab0847fcb7851704f86f88f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:40.847369 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-b9368fad-f310-49b5-9edd-d5e76d2c7481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:40.922835 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1115/2218] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:40 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:40.935265 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d9043-6488-4e6e-8d3b-e3c8c6f6ed37 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:40.937438 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-bf2d9043-6488-4e6e-8d3b-e3c8c6f6ed37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:41.004138 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d9043-6488-4e6e-8d3b-e3c8c6f6ed37 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:41.007946 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f6b01b-3864-437e-951f-4529d5f003d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:41.008556 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6b01b-3864-437e-951f-4529d5f003d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:41.008831 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6b01b-3864-437e-951f-4529d5f003d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:41.009099 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6b01b-3864-437e-951f-4529d5f003d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:41.019055 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f6b01b-3864-437e-951f-4529d5f003d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:41.020733 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f6b01b-3864-437e-951f-4529d5f003d6 service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:41.031452 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1104/2219] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:43:40 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:43:41.104106 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9368fad-f310-49b5-9edd-d5e76d2c7481 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:41.114337 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9368fad-f310-49b5-9edd-d5e76d2c7481 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:41.117931 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617103da-88ae-42fc-a733-05e3fd0106f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:41.118523 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617103da-88ae-42fc-a733-05e3fd0106f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:41.118797 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617103da-88ae-42fc-a733-05e3fd0106f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:41.119065 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617103da-88ae-42fc-a733-05e3fd0106f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:41.183310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617103da-88ae-42fc-a733-05e3fd0106f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:41.184799 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617103da-88ae-42fc-a733-05e3fd0106f8 service cinder] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:41.194153 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1116/2220] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:43:41 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:43:41.498944 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f6b01b-3864-437e-951f-4529d5f003d6 service cinder] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:41.562241 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f6b01b-3864-437e-951f-4529d5f003d6 service cinder] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:41.568783 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dda0b6-1803-48ca-a41f-1b740e21c26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:41.569595 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dda0b6-1803-48ca-a41f-1b740e21c26d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:41.569986 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dda0b6-1803-48ca-a41f-1b740e21c26d admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:41.570264 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dda0b6-1803-48ca-a41f-1b740e21c26d admin admin] PATH_INFO: `/v3/users/4ef76352db6645299c9e9488063af91c` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:41.571431 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dda0b6-1803-48ca-a41f-1b740e21c26d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ef76352db6645299c9e9488063af91c)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:41.573345 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dda0b6-1803-48ca-a41f-1b740e21c26d admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:41.587469 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:41.623818 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-79dda0b6-1803-48ca-a41f-1b740e21c26d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ef76352db6645299c9e9488063af91c'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:41.624526 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-79dda0b6-1803-48ca-a41f-1b740e21c26d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ef76352db6645299c9e9488063af91c'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:41.636778 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-79dda0b6-1803-48ca-a41f-1b740e21c26d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ef76352db6645299c9e9488063af91c'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:41.646164 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-79dda0b6-1803-48ca-a41f-1b740e21c26d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ef76352db6645299c9e9488063af91c'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:41.653807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-79dda0b6-1803-48ca-a41f-1b740e21c26d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ef76352db6645299c9e9488063af91c'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:41.660525 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1105/2221] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:43:41 2022] DELETE /identity/v3/users/4ef76352db6645299c9e9488063af91c => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:41.664728 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7328a54c-84e9-4cf5-ac88-ca33a06d28c6 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:41.665029 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7328a54c-84e9-4cf5-ac88-ca33a06d28c6 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:41.665285 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7328a54c-84e9-4cf5-ac88-ca33a06d28c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:41.685346 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-7328a54c-84e9-4cf5-ac88-ca33a06d28c6 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:41.686544 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-7328a54c-84e9-4cf5-ac88-ca33a06d28c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:41.705509 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:41.747107 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1117/2222] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:41.756854 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dda0b6-1803-48ca-a41f-1b740e21c26d admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:41.758249 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-79dda0b6-1803-48ca-a41f-1b740e21c26d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:41.816754 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dda0b6-1803-48ca-a41f-1b740e21c26d admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:41.820597 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0523dc0f-edf6-4824-946e-ac734f5e322a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:41.821200 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0523dc0f-edf6-4824-946e-ac734f5e322a service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:41.821477 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0523dc0f-edf6-4824-946e-ac734f5e322a service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:41.821747 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0523dc0f-edf6-4824-946e-ac734f5e322a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:41.831950 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0523dc0f-edf6-4824-946e-ac734f5e322a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:41.833585 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0523dc0f-edf6-4824-946e-ac734f5e322a service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:41.844334 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1106/2223] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:43:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:43:42.071593 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7328a54c-84e9-4cf5-ac88-ca33a06d28c6 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:42.081444 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-7328a54c-84e9-4cf5-ac88-ca33a06d28c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:42.099510 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7328a54c-84e9-4cf5-ac88-ca33a06d28c6 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:42.104809 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d3cef5-cb2b-4d2e-8926-285bed358b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:42.105657 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3cef5-cb2b-4d2e-8926-285bed358b67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:42.106061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3cef5-cb2b-4d2e-8926-285bed358b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:42.106522 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3cef5-cb2b-4d2e-8926-285bed358b67 admin admin] PATH_INFO: `/v3/users/8dfa19ff3652461d82c4d773304aee67` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:42.107923 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d3cef5-cb2b-4d2e-8926-285bed358b67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8dfa19ff3652461d82c4d773304aee67)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:42.110120 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d3cef5-cb2b-4d2e-8926-285bed358b67 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:42.178160 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-c1d3cef5-cb2b-4d2e-8926-285bed358b67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8dfa19ff3652461d82c4d773304aee67'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:42.178832 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-c1d3cef5-cb2b-4d2e-8926-285bed358b67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8dfa19ff3652461d82c4d773304aee67'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:42.194361 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0523dc0f-edf6-4824-946e-ac734f5e322a service neutron] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:42.195832 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-0523dc0f-edf6-4824-946e-ac734f5e322a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:42.198380 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-c1d3cef5-cb2b-4d2e-8926-285bed358b67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8dfa19ff3652461d82c4d773304aee67'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:42.207875 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-c1d3cef5-cb2b-4d2e-8926-285bed358b67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8dfa19ff3652461d82c4d773304aee67'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:42.208625 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-c1d3cef5-cb2b-4d2e-8926-285bed358b67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8dfa19ff3652461d82c4d773304aee67'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:42.233616 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1118/2224] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:43:42 2022] DELETE /identity/v3/users/8dfa19ff3652461d82c4d773304aee67 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:42.239481 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e142c975-ef71-44f8-b7f6-abcbedfe58bf None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:42.239816 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e142c975-ef71-44f8-b7f6-abcbedfe58bf None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:42.240559 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e142c975-ef71-44f8-b7f6-abcbedfe58bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:42.265885 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-e142c975-ef71-44f8-b7f6-abcbedfe58bf None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:42.267083 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-e142c975-ef71-44f8-b7f6-abcbedfe58bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:42.277217 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0523dc0f-edf6-4824-946e-ac734f5e322a service neutron] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:42.343326 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1119/2225] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:42.343663 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef1004f-70f6-46da-9ff1-793327bddf75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:42.344249 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e142c975-ef71-44f8-b7f6-abcbedfe58bf None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:42.344362 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1004f-70f6-46da-9ff1-793327bddf75 service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:42.344570 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1004f-70f6-46da-9ff1-793327bddf75 service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:42.344841 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1004f-70f6-46da-9ff1-793327bddf75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:42.347795 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-e142c975-ef71-44f8-b7f6-abcbedfe58bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:42.348265 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-cef1004f-70f6-46da-9ff1-793327bddf75 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:42.353594 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:42.360851 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e142c975-ef71-44f8-b7f6-abcbedfe58bf None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:42.365701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f1bc0-8a9d-4f89-9633-cf8adea59da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:42.366325 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f1bc0-8a9d-4f89-9633-cf8adea59da1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:42.366605 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f1bc0-8a9d-4f89-9633-cf8adea59da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:42.366906 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f1bc0-8a9d-4f89-9633-cf8adea59da1 admin admin] PATH_INFO: `/v3/projects/46b7868cbff043f3aa9072f795a0314b` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:42.368066 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50f1bc0-8a9d-4f89-9633-cf8adea59da1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46b7868cbff043f3aa9072f795a0314b)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:42.369962 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50f1bc0-8a9d-4f89-9633-cf8adea59da1 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:42.406756 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:42.415888 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef1004f-70f6-46da-9ff1-793327bddf75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:42.417316 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef1004f-70f6-46da-9ff1-793327bddf75 service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:42.421535 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-a50f1bc0-8a9d-4f89-9633-cf8adea59da1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46b7868cbff043f3aa9072f795a0314b', 'request_id': 'req-a50f1bc0-8a9d-4f89-9633-cf8adea59da1'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:42.427166 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-a50f1bc0-8a9d-4f89-9633-cf8adea59da1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46b7868cbff043f3aa9072f795a0314b', 'request_id': 'req-a50f1bc0-8a9d-4f89-9633-cf8adea59da1'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:42.429246 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1107/2226] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:43:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:43:42.429246 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef1004f-70f6-46da-9ff1-793327bddf75 service cinder] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:42.430241 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-cef1004f-70f6-46da-9ff1-793327bddf75 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:42.433966 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-a50f1bc0-8a9d-4f89-9633-cf8adea59da1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46b7868cbff043f3aa9072f795a0314b', 'request_id': 'req-a50f1bc0-8a9d-4f89-9633-cf8adea59da1'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:42.452414 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-a50f1bc0-8a9d-4f89-9633-cf8adea59da1 admin admin] The token cache is being invalidate because project 46b7868cbff043f3aa9072f795a0314b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:43:42.452892 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-a50f1bc0-8a9d-4f89-9633-cf8adea59da1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:42.454002 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-a50f1bc0-8a9d-4f89-9633-cf8adea59da1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:42.455925 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1120/2227] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:43:42 2022] DELETE /identity/v3/projects/46b7868cbff043f3aa9072f795a0314b => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:42.460216 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f1bc0-8a9d-4f89-9633-cf8adea59da1 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:42.463396 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-a50f1bc0-8a9d-4f89-9633-cf8adea59da1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:42.496003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef1004f-70f6-46da-9ff1-793327bddf75 service cinder] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:42.561661 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f1bc0-8a9d-4f89-9633-cf8adea59da1 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:42.566968 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c5b7ab-2f7f-48f8-9394-1afc92c31db7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:42.567630 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c5b7ab-2f7f-48f8-9394-1afc92c31db7 service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:42.567908 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c5b7ab-2f7f-48f8-9394-1afc92c31db7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:42.568176 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c5b7ab-2f7f-48f8-9394-1afc92c31db7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:42.569266 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2a00fc-8e49-4aa1-aad6-f5b43fba1c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:42.569914 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a00fc-8e49-4aa1-aad6-f5b43fba1c3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:42.570029 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-c1c5b7ab-2f7f-48f8-9394-1afc92c31db7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:42.570199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a00fc-8e49-4aa1-aad6-f5b43fba1c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:42.570479 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a00fc-8e49-4aa1-aad6-f5b43fba1c3f admin admin] PATH_INFO: `/v3/users/9f820d54ab0847fcb7851704f86f88f2` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:42.571754 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2a00fc-8e49-4aa1-aad6-f5b43fba1c3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f820d54ab0847fcb7851704f86f88f2)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:42.573800 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2a00fc-8e49-4aa1-aad6-f5b43fba1c3f admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:42.595686 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c5b7ab-2f7f-48f8-9394-1afc92c31db7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:42.597806 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c5b7ab-2f7f-48f8-9394-1afc92c31db7 service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:42.614336 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1108/2228] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:43:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:43:42.629634 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-fa2a00fc-8e49-4aa1-aad6-f5b43fba1c3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f820d54ab0847fcb7851704f86f88f2'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:42.630427 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-fa2a00fc-8e49-4aa1-aad6-f5b43fba1c3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f820d54ab0847fcb7851704f86f88f2'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:42.641070 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-fa2a00fc-8e49-4aa1-aad6-f5b43fba1c3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f820d54ab0847fcb7851704f86f88f2'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:42.651140 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-fa2a00fc-8e49-4aa1-aad6-f5b43fba1c3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f820d54ab0847fcb7851704f86f88f2'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:42.651903 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-fa2a00fc-8e49-4aa1-aad6-f5b43fba1c3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f820d54ab0847fcb7851704f86f88f2'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:42.684951 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1121/2229] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:43:42 2022] DELETE /identity/v3/users/9f820d54ab0847fcb7851704f86f88f2 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:43.131028 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c5b7ab-2f7f-48f8-9394-1afc92c31db7 service neutron] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:43.143460 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c5b7ab-2f7f-48f8-9394-1afc92c31db7 service neutron] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:43.255290 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2a00fc-8e49-4aa1-aad6-f5b43fba1c3f admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:43.257939 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-fa2a00fc-8e49-4aa1-aad6-f5b43fba1c3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:43.292747 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ada114-2789-430e-a0b8-c21ec9f78ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:43.293463 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ada114-2789-430e-a0b8-c21ec9f78ed8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:43.294141 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ada114-2789-430e-a0b8-c21ec9f78ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:43.295077 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ada114-2789-430e-a0b8-c21ec9f78ed8 admin admin] PATH_INFO: `/v3/projects/965739d6967747baaae58db1381688f6` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:43.296811 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ada114-2789-430e-a0b8-c21ec9f78ed8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=965739d6967747baaae58db1381688f6)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:43.299996 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ada114-2789-430e-a0b8-c21ec9f78ed8 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:43.311888 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:43.315756 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:43.339149 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2a00fc-8e49-4aa1-aad6-f5b43fba1c3f admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:43.345476 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba233945-0100-4255-810d-6ad1c8baa1d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:43.346484 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba233945-0100-4255-810d-6ad1c8baa1d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:43.347372 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba233945-0100-4255-810d-6ad1c8baa1d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:43.347982 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba233945-0100-4255-810d-6ad1c8baa1d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:43.351472 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-ba233945-0100-4255-810d-6ad1c8baa1d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:43.394939 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-d5ada114-2789-430e-a0b8-c21ec9f78ed8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '965739d6967747baaae58db1381688f6', 'request_id': 'req-d5ada114-2789-430e-a0b8-c21ec9f78ed8'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:43.413661 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-d5ada114-2789-430e-a0b8-c21ec9f78ed8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '965739d6967747baaae58db1381688f6', 'request_id': 'req-d5ada114-2789-430e-a0b8-c21ec9f78ed8'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:43.427955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-d5ada114-2789-430e-a0b8-c21ec9f78ed8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '965739d6967747baaae58db1381688f6', 'request_id': 'req-d5ada114-2789-430e-a0b8-c21ec9f78ed8'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:43.452106 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-d5ada114-2789-430e-a0b8-c21ec9f78ed8 admin admin] The token cache is being invalidate because project 965739d6967747baaae58db1381688f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:43:43.452420 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-d5ada114-2789-430e-a0b8-c21ec9f78ed8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:43.454058 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-d5ada114-2789-430e-a0b8-c21ec9f78ed8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:43.457013 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1109/2230] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:43:43 2022] DELETE /identity/v3/projects/965739d6967747baaae58db1381688f6 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:43.530332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba233945-0100-4255-810d-6ad1c8baa1d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:43.532885 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba233945-0100-4255-810d-6ad1c8baa1d9 service cinder] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:43.547672 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1122/2231] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:43:43 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:43:43.634299 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ada114-2789-430e-a0b8-c21ec9f78ed8 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:43.637262 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-d5ada114-2789-430e-a0b8-c21ec9f78ed8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:43.745045 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ada114-2789-430e-a0b8-c21ec9f78ed8 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:43.750519 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be892010-d048-44ef-b060-c085d93b1632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:43.751287 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be892010-d048-44ef-b060-c085d93b1632 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:43.751669 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be892010-d048-44ef-b060-c085d93b1632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:43.752008 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be892010-d048-44ef-b060-c085d93b1632 admin admin] PATH_INFO: `/v3/projects/54444cd4cd134a59bec2232c26310a28` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:43.753183 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be892010-d048-44ef-b060-c085d93b1632 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54444cd4cd134a59bec2232c26310a28)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:43.755158 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be892010-d048-44ef-b060-c085d93b1632 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:43.806547 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-be892010-d048-44ef-b060-c085d93b1632 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54444cd4cd134a59bec2232c26310a28', 'request_id': 'req-be892010-d048-44ef-b060-c085d93b1632'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:43.813552 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-be892010-d048-44ef-b060-c085d93b1632 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54444cd4cd134a59bec2232c26310a28', 'request_id': 'req-be892010-d048-44ef-b060-c085d93b1632'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:43.820275 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-be892010-d048-44ef-b060-c085d93b1632 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54444cd4cd134a59bec2232c26310a28', 'request_id': 'req-be892010-d048-44ef-b060-c085d93b1632'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:43.833028 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-be892010-d048-44ef-b060-c085d93b1632 admin admin] The token cache is being invalidate because project 54444cd4cd134a59bec2232c26310a28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:43:43.833320 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-be892010-d048-44ef-b060-c085d93b1632 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:43.833964 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-be892010-d048-44ef-b060-c085d93b1632 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:43.835571 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1110/2232] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:43:43 2022] DELETE /identity/v3/projects/54444cd4cd134a59bec2232c26310a28 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:43.839936 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba233945-0100-4255-810d-6ad1c8baa1d9 service cinder] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:43.842094 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-ba233945-0100-4255-810d-6ad1c8baa1d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:43.858073 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba233945-0100-4255-810d-6ad1c8baa1d9 service cinder] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:43.916927 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86beef47-dea3-44cb-9440-517be6d51df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:43.917544 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86beef47-dea3-44cb-9440-517be6d51df6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:43.917821 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86beef47-dea3-44cb-9440-517be6d51df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:43.918087 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86beef47-dea3-44cb-9440-517be6d51df6 admin admin] PATH_INFO: `/v3/users/a650b7c058a84a0885f002d78dd90c66` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:43.920186 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86beef47-dea3-44cb-9440-517be6d51df6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a650b7c058a84a0885f002d78dd90c66)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:43.922143 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86beef47-dea3-44cb-9440-517be6d51df6 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:43.972681 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-86beef47-dea3-44cb-9440-517be6d51df6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a650b7c058a84a0885f002d78dd90c66'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:43.973339 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-86beef47-dea3-44cb-9440-517be6d51df6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a650b7c058a84a0885f002d78dd90c66'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:43.981978 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-86beef47-dea3-44cb-9440-517be6d51df6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a650b7c058a84a0885f002d78dd90c66'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:43.989445 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-86beef47-dea3-44cb-9440-517be6d51df6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a650b7c058a84a0885f002d78dd90c66'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:43.990350 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-86beef47-dea3-44cb-9440-517be6d51df6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a650b7c058a84a0885f002d78dd90c66'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:44.008917 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1123/2233] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:43:43 2022] DELETE /identity/v3/users/a650b7c058a84a0885f002d78dd90c66 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:43:44.305744 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be892010-d048-44ef-b060-c085d93b1632 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:44.307227 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-be892010-d048-44ef-b060-c085d93b1632 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:44.355050 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be892010-d048-44ef-b060-c085d93b1632 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:44.383367 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:44.437925 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9d7c6f-c743-414c-a7ea-6c653d4f19cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:44.438295 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d7c6f-c743-414c-a7ea-6c653d4f19cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:44.438570 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d7c6f-c743-414c-a7ea-6c653d4f19cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:44.438838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d7c6f-c743-414c-a7ea-6c653d4f19cf admin admin] PATH_INFO: `/v3/projects/8a1039d4f2324fcc82415d52d04bc527` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:44.440195 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9d7c6f-c743-414c-a7ea-6c653d4f19cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a1039d4f2324fcc82415d52d04bc527)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:44.442639 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9d7c6f-c743-414c-a7ea-6c653d4f19cf admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:44.521168 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-5d9d7c6f-c743-414c-a7ea-6c653d4f19cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a1039d4f2324fcc82415d52d04bc527', 'request_id': 'req-5d9d7c6f-c743-414c-a7ea-6c653d4f19cf'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:44.527121 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-5d9d7c6f-c743-414c-a7ea-6c653d4f19cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a1039d4f2324fcc82415d52d04bc527', 'request_id': 'req-5d9d7c6f-c743-414c-a7ea-6c653d4f19cf'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:44.544471 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-5d9d7c6f-c743-414c-a7ea-6c653d4f19cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a1039d4f2324fcc82415d52d04bc527', 'request_id': 'req-5d9d7c6f-c743-414c-a7ea-6c653d4f19cf'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:44.569123 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-5d9d7c6f-c743-414c-a7ea-6c653d4f19cf admin admin] The token cache is being invalidate because project 8a1039d4f2324fcc82415d52d04bc527 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:43:44.569314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-5d9d7c6f-c743-414c-a7ea-6c653d4f19cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:44.569935 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-5d9d7c6f-c743-414c-a7ea-6c653d4f19cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:44.580841 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1111/2234] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:43:44 2022] DELETE /identity/v3/projects/8a1039d4f2324fcc82415d52d04bc527 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:44.581031 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09116fc-d068-465f-8115-7f57893851a2 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:44.581475 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09116fc-d068-465f-8115-7f57893851a2 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:44.581574 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09116fc-d068-465f-8115-7f57893851a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:44.603569 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-d09116fc-d068-465f-8115-7f57893851a2 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:44.604860 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-d09116fc-d068-465f-8115-7f57893851a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:44.610961 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:44.710642 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1124/2235] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:43:44.720222 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e273ed-00f7-4438-b4b3-748fb5320e11 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:44.721356 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e273ed-00f7-4438-b4b3-748fb5320e11 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:44.721356 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e273ed-00f7-4438-b4b3-748fb5320e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:44.752219 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-29e273ed-00f7-4438-b4b3-748fb5320e11 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:44.753657 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-29e273ed-00f7-4438-b4b3-748fb5320e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:44.767459 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1112/2236] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:44.771875 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09116fc-d068-465f-8115-7f57893851a2 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:44.790996 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09116fc-d068-465f-8115-7f57893851a2 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:44.800287 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a82f258-50d6-4583-9380-17fad7b2620b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:44.801127 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a82f258-50d6-4583-9380-17fad7b2620b admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:44.801546 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a82f258-50d6-4583-9380-17fad7b2620b admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:44.801981 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a82f258-50d6-4583-9380-17fad7b2620b admin admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:44.802917 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a82f258-50d6-4583-9380-17fad7b2620b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:43:44.803536 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a82f258-50d6-4583-9380-17fad7b2620b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:44.805577 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a82f258-50d6-4583-9380-17fad7b2620b admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:44.823437 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1125/2237] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 14:43:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:43:44.824525 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e273ed-00f7-4438-b4b3-748fb5320e11 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:44.834711 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e273ed-00f7-4438-b4b3-748fb5320e11 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:44.842924 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1632eeca-1591-44b8-8e8e-19aafbad0e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:44.843574 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1632eeca-1591-44b8-8e8e-19aafbad0e19 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:44.843857 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1632eeca-1591-44b8-8e8e-19aafbad0e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:44.844128 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1632eeca-1591-44b8-8e8e-19aafbad0e19 admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:44.844779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1632eeca-1591-44b8-8e8e-19aafbad0e19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:44.846811 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1632eeca-1591-44b8-8e8e-19aafbad0e19 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:44.882586 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1113/2238] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:43:44 2022] POST /identity/v3/projects => generated 397 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:43:44.886711 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a82f258-50d6-4583-9380-17fad7b2620b admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:44.900357 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a82f258-50d6-4583-9380-17fad7b2620b admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:44.972084 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b0aac4-8dfd-4d89-bf1e-4ecde3b4092b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:44.972702 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b0aac4-8dfd-4d89-bf1e-4ecde3b4092b admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:44.973019 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b0aac4-8dfd-4d89-bf1e-4ecde3b4092b admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:44.973299 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b0aac4-8dfd-4d89-bf1e-4ecde3b4092b admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:44.973990 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b0aac4-8dfd-4d89-bf1e-4ecde3b4092b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:44.976166 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b0aac4-8dfd-4d89-bf1e-4ecde3b4092b admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:45.008431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1126/2239] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:43:44 2022] POST /identity/v3/users => generated 299 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:45.014236 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1632eeca-1591-44b8-8e8e-19aafbad0e19 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:45.030226 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1632eeca-1591-44b8-8e8e-19aafbad0e19 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:45.036099 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3630ab-f857-4e29-94f3-484b355a80e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:45.037003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3630ab-f857-4e29-94f3-484b355a80e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:45.037292 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3630ab-f857-4e29-94f3-484b355a80e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:45.037842 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3630ab-f857-4e29-94f3-484b355a80e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:45.038232 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3630ab-f857-4e29-94f3-484b355a80e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:45.040110 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3630ab-f857-4e29-94f3-484b355a80e1 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:45.052017 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1114/2240] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:45.056217 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b0aac4-8dfd-4d89-bf1e-4ecde3b4092b admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:45.072538 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b0aac4-8dfd-4d89-bf1e-4ecde3b4092b admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:45.096039 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5455c067-71b6-486f-9fc6-0959e6a80bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:45.096792 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455c067-71b6-486f-9fc6-0959e6a80bfc admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:45.097355 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455c067-71b6-486f-9fc6-0959e6a80bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:45.097503 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455c067-71b6-486f-9fc6-0959e6a80bfc admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:45.098179 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5455c067-71b6-486f-9fc6-0959e6a80bfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:45.099805 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5455c067-71b6-486f-9fc6-0959e6a80bfc admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:45.116014 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1127/2241] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:45.120444 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3630ab-f857-4e29-94f3-484b355a80e1 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:45.131646 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3630ab-f857-4e29-94f3-484b355a80e1 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:45.145514 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e073ff-4096-424c-b8e4-584f95899bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:45.148646 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e073ff-4096-424c-b8e4-584f95899bc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:45.149188 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e073ff-4096-424c-b8e4-584f95899bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:45.150127 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e073ff-4096-424c-b8e4-584f95899bc9 admin admin] PATH_INFO: `/v3/projects/bdbc620c36b34e79ab20f16a08721b89/users/f254345518c44cc295233adfec37b93c/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:45.197589 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e073ff-4096-424c-b8e4-584f95899bc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdbc620c36b34e79ab20f16a08721b89, user_id=f254345518c44cc295233adfec37b93c, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:45.201109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e073ff-4096-424c-b8e4-584f95899bc9 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:45.229230 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1115/2242] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:45 2022] PUT /identity/v3/projects/bdbc620c36b34e79ab20f16a08721b89/users/f254345518c44cc295233adfec37b93c/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:45.242308 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675890b-de32-4ac6-be29-8fe0838a5a6e None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:45.242819 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675890b-de32-4ac6-be29-8fe0838a5a6e None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:45.243186 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675890b-de32-4ac6-be29-8fe0838a5a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:45.275948 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-f675890b-de32-4ac6-be29-8fe0838a5a6e None None] MFA Rules not processed for user `f254345518c44cc295233adfec37b93c`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:45.277267 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-f675890b-de32-4ac6-be29-8fe0838a5a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:45.353903 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1128/2243] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:45 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:45.366499 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e073ff-4096-424c-b8e4-584f95899bc9 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:45.369109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-63e073ff-4096-424c-b8e4-584f95899bc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:45.442918 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e073ff-4096-424c-b8e4-584f95899bc9 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:45.447470 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb4872-6932-4217-a1b4-195cedf42745 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:45.448280 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb4872-6932-4217-a1b4-195cedf42745 service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:45.448558 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb4872-6932-4217-a1b4-195cedf42745 service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:45.448835 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb4872-6932-4217-a1b4-195cedf42745 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:45.459266 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:45.464734 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcb4872-6932-4217-a1b4-195cedf42745 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:45.467142 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcb4872-6932-4217-a1b4-195cedf42745 service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:45.479426 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1116/2244] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:43:45 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:43:47.127758 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f675890b-de32-4ac6-be29-8fe0838a5a6e None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:47.129186 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-f675890b-de32-4ac6-be29-8fe0838a5a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:47.144350 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:47.270379 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f675890b-de32-4ac6-be29-8fe0838a5a6e None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:47.275448 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cebc89-35ab-4444-9352-919c5cefd593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:47.276440 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cebc89-35ab-4444-9352-919c5cefd593 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:47.276891 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cebc89-35ab-4444-9352-919c5cefd593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:47.277235 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cebc89-35ab-4444-9352-919c5cefd593 admin admin] PATH_INFO: `/v3/users/0642fa92a86c4e01a42bf087cd36cace` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:47.279457 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cebc89-35ab-4444-9352-919c5cefd593 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0642fa92a86c4e01a42bf087cd36cace)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:47.282519 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cebc89-35ab-4444-9352-919c5cefd593 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:47.344002 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-d3cebc89-35ab-4444-9352-919c5cefd593 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0642fa92a86c4e01a42bf087cd36cace'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:47.344705 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-d3cebc89-35ab-4444-9352-919c5cefd593 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0642fa92a86c4e01a42bf087cd36cace'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:47.353265 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-d3cebc89-35ab-4444-9352-919c5cefd593 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0642fa92a86c4e01a42bf087cd36cace'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:47.361008 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-d3cebc89-35ab-4444-9352-919c5cefd593 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0642fa92a86c4e01a42bf087cd36cace'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:47.361773 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-d3cebc89-35ab-4444-9352-919c5cefd593 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0642fa92a86c4e01a42bf087cd36cace'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:47.379785 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1129/2245] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:43:47 2022] DELETE /identity/v3/users/0642fa92a86c4e01a42bf087cd36cace => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:47.578309 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb4872-6932-4217-a1b4-195cedf42745 service cinder] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:47.609156 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb4872-6932-4217-a1b4-195cedf42745 service cinder] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:47.670616 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4be767e-9a3a-47f6-abf2-c3a3d7971a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:47.671163 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be767e-9a3a-47f6-abf2-c3a3d7971a47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:47.671478 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be767e-9a3a-47f6-abf2-c3a3d7971a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:47.671746 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be767e-9a3a-47f6-abf2-c3a3d7971a47 admin admin] PATH_INFO: `/v3/projects/8aeffb40eafc44639c8e5032dce345aa` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:47.672862 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4be767e-9a3a-47f6-abf2-c3a3d7971a47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8aeffb40eafc44639c8e5032dce345aa)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:47.674740 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4be767e-9a3a-47f6-abf2-c3a3d7971a47 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:47.723624 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b4be767e-9a3a-47f6-abf2-c3a3d7971a47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8aeffb40eafc44639c8e5032dce345aa', 'request_id': 'req-b4be767e-9a3a-47f6-abf2-c3a3d7971a47'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:47.740076 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b4be767e-9a3a-47f6-abf2-c3a3d7971a47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8aeffb40eafc44639c8e5032dce345aa', 'request_id': 'req-b4be767e-9a3a-47f6-abf2-c3a3d7971a47'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:47.747283 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b4be767e-9a3a-47f6-abf2-c3a3d7971a47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8aeffb40eafc44639c8e5032dce345aa', 'request_id': 'req-b4be767e-9a3a-47f6-abf2-c3a3d7971a47'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:47.761420 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b4be767e-9a3a-47f6-abf2-c3a3d7971a47 admin admin] The token cache is being invalidate because project 8aeffb40eafc44639c8e5032dce345aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:43:47.761710 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b4be767e-9a3a-47f6-abf2-c3a3d7971a47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:47.762359 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b4be767e-9a3a-47f6-abf2-c3a3d7971a47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:47.764155 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1117/2246] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:43:47 2022] DELETE /identity/v3/projects/8aeffb40eafc44639c8e5032dce345aa => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:47.769178 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6971f146-3b34-4dc6-b985-8de923bf2210 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:47.769473 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6971f146-3b34-4dc6-b985-8de923bf2210 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:47.769778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6971f146-3b34-4dc6-b985-8de923bf2210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:47.770594 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4be767e-9a3a-47f6-abf2-c3a3d7971a47 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:47.772063 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-b4be767e-9a3a-47f6-abf2-c3a3d7971a47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:47.789942 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-6971f146-3b34-4dc6-b985-8de923bf2210 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:47.791137 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-6971f146-3b34-4dc6-b985-8de923bf2210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:47.838355 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4be767e-9a3a-47f6-abf2-c3a3d7971a47 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:47.843983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f1e91d-9a48-44f2-a639-c3abd9959b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:47.844607 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f1e91d-9a48-44f2-a639-c3abd9959b73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:47.844889 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f1e91d-9a48-44f2-a639-c3abd9959b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:47.845162 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f1e91d-9a48-44f2-a639-c3abd9959b73 admin admin] PATH_INFO: `/v3/users/a96cf3acb4d54113bb4706a21bdb8486` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:47.846300 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f1e91d-9a48-44f2-a639-c3abd9959b73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a96cf3acb4d54113bb4706a21bdb8486)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:47.848254 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f1e91d-9a48-44f2-a639-c3abd9959b73 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:47.852986 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1130/2247] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:43:47.861340 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f76171-b3b2-4a4b-be9e-4d71adfce231 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:47.861603 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f76171-b3b2-4a4b-be9e-4d71adfce231 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:47.861881 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f76171-b3b2-4a4b-be9e-4d71adfce231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:47.877305 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:47.881738 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-15f76171-b3b2-4a4b-be9e-4d71adfce231 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:47.882839 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-15f76171-b3b2-4a4b-be9e-4d71adfce231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:47.891536 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1131/2248] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:43:47.895643 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f76171-b3b2-4a4b-be9e-4d71adfce231 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:47.897581 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b0f1e91d-9a48-44f2-a639-c3abd9959b73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a96cf3acb4d54113bb4706a21bdb8486'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:47.898231 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b0f1e91d-9a48-44f2-a639-c3abd9959b73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a96cf3acb4d54113bb4706a21bdb8486'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:47.905043 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f76171-b3b2-4a4b-be9e-4d71adfce231 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:47.909653 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e9d892-1160-4abe-b9a2-0b5b42255442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:47.910231 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e9d892-1160-4abe-b9a2-0b5b42255442 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:47.910511 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e9d892-1160-4abe-b9a2-0b5b42255442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:47.910780 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e9d892-1160-4abe-b9a2-0b5b42255442 admin admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:47.911379 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b0f1e91d-9a48-44f2-a639-c3abd9959b73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a96cf3acb4d54113bb4706a21bdb8486'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:47.911514 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e9d892-1160-4abe-b9a2-0b5b42255442 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:43:47.911722 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e9d892-1160-4abe-b9a2-0b5b42255442 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:47.913328 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e9d892-1160-4abe-b9a2-0b5b42255442 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:47.919443 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b0f1e91d-9a48-44f2-a639-c3abd9959b73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a96cf3acb4d54113bb4706a21bdb8486'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:47.925699 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1132/2249] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 14:43:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:43:47.927280 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b0f1e91d-9a48-44f2-a639-c3abd9959b73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a96cf3acb4d54113bb4706a21bdb8486'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:47.929866 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e9d892-1160-4abe-b9a2-0b5b42255442 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:47.931826 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-b1e9d892-1160-4abe-b9a2-0b5b42255442 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:47.933384 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1118/2250] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:43:47 2022] DELETE /identity/v3/users/a96cf3acb4d54113bb4706a21bdb8486 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:47.937565 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860130fe-f2fb-4ac9-aeab-c9a5ecf46110 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:47.937852 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860130fe-f2fb-4ac9-aeab-c9a5ecf46110 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:47.938126 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860130fe-f2fb-4ac9-aeab-c9a5ecf46110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:47.942192 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:47.946388 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e9d892-1160-4abe-b9a2-0b5b42255442 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:47.959783 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-860130fe-f2fb-4ac9-aeab-c9a5ecf46110 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:47.961001 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-860130fe-f2fb-4ac9-aeab-c9a5ecf46110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:48.010512 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed3e0f9-e12a-495c-9893-35cb4bfdebc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:48.011114 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3e0f9-e12a-495c-9893-35cb4bfdebc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:48.011424 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3e0f9-e12a-495c-9893-35cb4bfdebc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:48.011801 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3e0f9-e12a-495c-9893-35cb4bfdebc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:48.013140 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed3e0f9-e12a-495c-9893-35cb4bfdebc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:48.015095 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed3e0f9-e12a-495c-9893-35cb4bfdebc0 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:48.021181 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1119/2251] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:43:48.032006 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860130fe-f2fb-4ac9-aeab-c9a5ecf46110 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:48.033389 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-860130fe-f2fb-4ac9-aeab-c9a5ecf46110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:48.043770 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1133/2252] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:43:47 2022] POST /identity/v3/projects => generated 373 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:48.047439 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed3e0f9-e12a-495c-9893-35cb4bfdebc0 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:48.096520 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860130fe-f2fb-4ac9-aeab-c9a5ecf46110 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:48.110129 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed3e0f9-e12a-495c-9893-35cb4bfdebc0 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:48.114809 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06f0ddd-f8f9-4c6f-92dc-6bf0330d0d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:48.115437 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f0ddd-f8f9-4c6f-92dc-6bf0330d0d0a admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:48.115711 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f0ddd-f8f9-4c6f-92dc-6bf0330d0d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:48.116007 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f0ddd-f8f9-4c6f-92dc-6bf0330d0d0a admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:48.116630 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06f0ddd-f8f9-4c6f-92dc-6bf0330d0d0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:48.118528 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06f0ddd-f8f9-4c6f-92dc-6bf0330d0d0a admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:48.141593 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1134/2253] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:43:48 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:43:48.144905 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae4d203-9947-4e23-a6ca-0eda25fd9252 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:48.145771 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4d203-9947-4e23-a6ca-0eda25fd9252 service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:48.145905 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06f0ddd-f8f9-4c6f-92dc-6bf0330d0d0a admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:48.146000 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4d203-9947-4e23-a6ca-0eda25fd9252 service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:48.146099 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4d203-9947-4e23-a6ca-0eda25fd9252 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:48.155834 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae4d203-9947-4e23-a6ca-0eda25fd9252 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:48.156385 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06f0ddd-f8f9-4c6f-92dc-6bf0330d0d0a admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:48.157247 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae4d203-9947-4e23-a6ca-0eda25fd9252 service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:48.161650 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2343bd-d76a-4267-a254-c8dcec254893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:48.162271 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2343bd-d76a-4267-a254-c8dcec254893 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:48.162555 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2343bd-d76a-4267-a254-c8dcec254893 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:48.162828 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2343bd-d76a-4267-a254-c8dcec254893 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:48.163501 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2343bd-d76a-4267-a254-c8dcec254893 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:48.165097 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2343bd-d76a-4267-a254-c8dcec254893 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:48.166951 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1120/2254] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:43:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:43:48.179537 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1135/2255] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:43:48.180625 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae4d203-9947-4e23-a6ca-0eda25fd9252 service neutron] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:48.185550 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:48.191109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae4d203-9947-4e23-a6ca-0eda25fd9252 service neutron] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:48.197049 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19533ce7-13d7-4ddd-8bcb-08f37bd1a3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:48.197684 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19533ce7-13d7-4ddd-8bcb-08f37bd1a3c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:48.197971 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19533ce7-13d7-4ddd-8bcb-08f37bd1a3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:48.198445 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19533ce7-13d7-4ddd-8bcb-08f37bd1a3c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:48.199569 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19533ce7-13d7-4ddd-8bcb-08f37bd1a3c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:48.201993 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19533ce7-13d7-4ddd-8bcb-08f37bd1a3c8 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:48.215281 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1121/2256] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:48.218500 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2343bd-d76a-4267-a254-c8dcec254893 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:48.229696 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2343bd-d76a-4267-a254-c8dcec254893 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:48.235818 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7117f2d-03b0-44a9-8055-bf6cfb5d96e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:48.236890 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7117f2d-03b0-44a9-8055-bf6cfb5d96e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:48.237246 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7117f2d-03b0-44a9-8055-bf6cfb5d96e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:48.237836 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7117f2d-03b0-44a9-8055-bf6cfb5d96e4 admin admin] PATH_INFO: `/v3/projects/94413b61a4d34324be36806a1a09ede3/users/b40ccb34a65e4c73820760918d7ddbd7/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:48.256173 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7117f2d-03b0-44a9-8055-bf6cfb5d96e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94413b61a4d34324be36806a1a09ede3, user_id=b40ccb34a65e4c73820760918d7ddbd7, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:48.259041 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7117f2d-03b0-44a9-8055-bf6cfb5d96e4 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:48.273813 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1136/2257] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:48 2022] PUT /identity/v3/projects/94413b61a4d34324be36806a1a09ede3/users/b40ccb34a65e4c73820760918d7ddbd7/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:48.285031 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258fe6c3-394c-4f8a-ba24-ef9e3c9aeae5 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:48.285031 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258fe6c3-394c-4f8a-ba24-ef9e3c9aeae5 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:48.285297 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258fe6c3-394c-4f8a-ba24-ef9e3c9aeae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:48.315682 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-258fe6c3-394c-4f8a-ba24-ef9e3c9aeae5 None None] MFA Rules not processed for user `b40ccb34a65e4c73820760918d7ddbd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:48.316957 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-258fe6c3-394c-4f8a-ba24-ef9e3c9aeae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:48.406978 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1122/2258] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:48.407804 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7117f2d-03b0-44a9-8055-bf6cfb5d96e4 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:48.459513 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:48.467644 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258fe6c3-394c-4f8a-ba24-ef9e3c9aeae5 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:48.469107 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-258fe6c3-394c-4f8a-ba24-ef9e3c9aeae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:48.486647 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7117f2d-03b0-44a9-8055-bf6cfb5d96e4 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:48.492676 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70270c77-0761-47cd-af5d-c022162dc0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:48.493662 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70270c77-0761-47cd-af5d-c022162dc0ff admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:48.493662 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70270c77-0761-47cd-af5d-c022162dc0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:48.493915 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70270c77-0761-47cd-af5d-c022162dc0ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:48.494563 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70270c77-0761-47cd-af5d-c022162dc0ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:48.497866 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70270c77-0761-47cd-af5d-c022162dc0ff admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:48.530085 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1137/2259] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:43:48 2022] POST /identity/v3/projects => generated 371 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:48.534081 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70270c77-0761-47cd-af5d-c022162dc0ff admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:48.554998 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258fe6c3-394c-4f8a-ba24-ef9e3c9aeae5 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:48.598208 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70270c77-0761-47cd-af5d-c022162dc0ff admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:48.602947 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b86c72-12c3-43c0-bb42-e301d0df9670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:48.603669 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b86c72-12c3-43c0-bb42-e301d0df9670 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:48.603868 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b86c72-12c3-43c0-bb42-e301d0df9670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:48.604143 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b86c72-12c3-43c0-bb42-e301d0df9670 admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:48.604782 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b86c72-12c3-43c0-bb42-e301d0df9670 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:48.607314 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b86c72-12c3-43c0-bb42-e301d0df9670 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:48.614283 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beca0756-bd49-4057-b3d0-67e58c16528e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:48.614922 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca0756-bd49-4057-b3d0-67e58c16528e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:48.615234 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca0756-bd49-4057-b3d0-67e58c16528e admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:48.615506 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca0756-bd49-4057-b3d0-67e58c16528e admin admin] PATH_INFO: `/v3/projects/1b1a27d46efd46bea9140505a4aab0cf` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:48.616654 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beca0756-bd49-4057-b3d0-67e58c16528e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b1a27d46efd46bea9140505a4aab0cf)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:48.618897 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beca0756-bd49-4057-b3d0-67e58c16528e admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:48.633911 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1138/2260] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:43:48 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:43:48.636345 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b86c72-12c3-43c0-bb42-e301d0df9670 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:48.647545 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b86c72-12c3-43c0-bb42-e301d0df9670 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:48.653276 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d267fe9-fc23-4b86-9f53-5a2407c4b39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:48.653951 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d267fe9-fc23-4b86-9f53-5a2407c4b39c admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:48.654275 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d267fe9-fc23-4b86-9f53-5a2407c4b39c admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:48.654552 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d267fe9-fc23-4b86-9f53-5a2407c4b39c admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:48.655198 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d267fe9-fc23-4b86-9f53-5a2407c4b39c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:48.656986 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d267fe9-fc23-4b86-9f53-5a2407c4b39c admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:48.665890 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:48.668183 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1139/2261] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:48.673168 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d267fe9-fc23-4b86-9f53-5a2407c4b39c admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:48.680461 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-beca0756-bd49-4057-b3d0-67e58c16528e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b1a27d46efd46bea9140505a4aab0cf', 'request_id': 'req-beca0756-bd49-4057-b3d0-67e58c16528e'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:48.683241 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d267fe9-fc23-4b86-9f53-5a2407c4b39c admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:48.686737 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-beca0756-bd49-4057-b3d0-67e58c16528e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b1a27d46efd46bea9140505a4aab0cf', 'request_id': 'req-beca0756-bd49-4057-b3d0-67e58c16528e'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:48.693368 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-beca0756-bd49-4057-b3d0-67e58c16528e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b1a27d46efd46bea9140505a4aab0cf', 'request_id': 'req-beca0756-bd49-4057-b3d0-67e58c16528e'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:48.706214 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-beca0756-bd49-4057-b3d0-67e58c16528e admin admin] The token cache is being invalidate because project 1b1a27d46efd46bea9140505a4aab0cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:43:48.706508 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-beca0756-bd49-4057-b3d0-67e58c16528e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:48.707142 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-beca0756-bd49-4057-b3d0-67e58c16528e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:48.708731 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1123/2262] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:43:48 2022] DELETE /identity/v3/projects/1b1a27d46efd46bea9140505a4aab0cf => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:43:48.712764 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beca0756-bd49-4057-b3d0-67e58c16528e admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:48.714041 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-beca0756-bd49-4057-b3d0-67e58c16528e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:48.749637 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24354734-5783-4211-9c4b-9d96b06c8619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:48.750327 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24354734-5783-4211-9c4b-9d96b06c8619 admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:48.750608 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24354734-5783-4211-9c4b-9d96b06c8619 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:48.750881 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24354734-5783-4211-9c4b-9d96b06c8619 admin admin] PATH_INFO: `/v3/projects/894adc4e436f49f58315b9923c5a2563/users/691e3af987ba4971a7af24efa11c75d2/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:48.767914 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24354734-5783-4211-9c4b-9d96b06c8619 admin admin] RBAC: Authorizing `identity:create_grant(project_id=894adc4e436f49f58315b9923c5a2563, user_id=691e3af987ba4971a7af24efa11c75d2, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:48.770744 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24354734-5783-4211-9c4b-9d96b06c8619 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:48.782679 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beca0756-bd49-4057-b3d0-67e58c16528e admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:48.783659 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1140/2263] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:48 2022] PUT /identity/v3/projects/894adc4e436f49f58315b9923c5a2563/users/691e3af987ba4971a7af24efa11c75d2/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:43:48.787672 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24354734-5783-4211-9c4b-9d96b06c8619 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:48.789276 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-24354734-5783-4211-9c4b-9d96b06c8619 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:48.824230 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:48.839995 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b3a617-7e66-4df7-aee2-0a0614c9a691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:48.841494 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3a617-7e66-4df7-aee2-0a0614c9a691 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:48.841887 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3a617-7e66-4df7-aee2-0a0614c9a691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:48.842162 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3a617-7e66-4df7-aee2-0a0614c9a691 admin admin] PATH_INFO: `/v3/users/2224ce98ffa242d79e7fcb8e183be47a` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:48.843353 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b3a617-7e66-4df7-aee2-0a0614c9a691 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2224ce98ffa242d79e7fcb8e183be47a)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:48.845268 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b3a617-7e66-4df7-aee2-0a0614c9a691 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:48.857531 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24354734-5783-4211-9c4b-9d96b06c8619 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:48.862428 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fef7c2-1212-4fda-a7fc-7af9d3c82918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:48.863107 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fef7c2-1212-4fda-a7fc-7af9d3c82918 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:48.863474 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fef7c2-1212-4fda-a7fc-7af9d3c82918 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:48.863820 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fef7c2-1212-4fda-a7fc-7af9d3c82918 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:48.864531 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fef7c2-1212-4fda-a7fc-7af9d3c82918 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:48.866149 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fef7c2-1212-4fda-a7fc-7af9d3c82918 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:48.877383 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1141/2264] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:48 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:43:48.881768 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fef7c2-1212-4fda-a7fc-7af9d3c82918 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:48.892160 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fef7c2-1212-4fda-a7fc-7af9d3c82918 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:48.893176 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-c2b3a617-7e66-4df7-aee2-0a0614c9a691 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2224ce98ffa242d79e7fcb8e183be47a'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:48.893845 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-c2b3a617-7e66-4df7-aee2-0a0614c9a691 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2224ce98ffa242d79e7fcb8e183be47a'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:48.897076 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1f0743-0d9c-4c7c-bcaa-b6ccdae595c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:48.897825 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1f0743-0d9c-4c7c-bcaa-b6ccdae595c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:48.898105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1f0743-0d9c-4c7c-bcaa-b6ccdae595c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:48.898380 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1f0743-0d9c-4c7c-bcaa-b6ccdae595c6 admin admin] PATH_INFO: `/v3/projects/894adc4e436f49f58315b9923c5a2563/users/691e3af987ba4971a7af24efa11c75d2/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:48.900322 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1f0743-0d9c-4c7c-bcaa-b6ccdae595c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=894adc4e436f49f58315b9923c5a2563, user_id=691e3af987ba4971a7af24efa11c75d2, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:48.903024 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1f0743-0d9c-4c7c-bcaa-b6ccdae595c6 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:48.906126 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-c2b3a617-7e66-4df7-aee2-0a0614c9a691 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2224ce98ffa242d79e7fcb8e183be47a'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:48.908279 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:48.913629 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1142/2265] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:48 2022] PUT /identity/v3/projects/894adc4e436f49f58315b9923c5a2563/users/691e3af987ba4971a7af24efa11c75d2/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:43:48.914231 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-c2b3a617-7e66-4df7-aee2-0a0614c9a691 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2224ce98ffa242d79e7fcb8e183be47a'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:48.925107 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-c2b3a617-7e66-4df7-aee2-0a0614c9a691 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2224ce98ffa242d79e7fcb8e183be47a'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:48.926649 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5312fbbb-186d-4d94-bac3-51427c5249a1 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:48.926881 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5312fbbb-186d-4d94-bac3-51427c5249a1 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:48.927182 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5312fbbb-186d-4d94-bac3-51427c5249a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:48.931783 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1124/2266] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:43:48 2022] DELETE /identity/v3/users/2224ce98ffa242d79e7fcb8e183be47a => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:43:48.957367 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-5312fbbb-186d-4d94-bac3-51427c5249a1 None None] MFA Rules not processed for user `691e3af987ba4971a7af24efa11c75d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:48.958598 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-5312fbbb-186d-4d94-bac3-51427c5249a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:49.032591 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1143/2267] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:48 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:49.045220 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b3a617-7e66-4df7-aee2-0a0614c9a691 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:49.046641 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-c2b3a617-7e66-4df7-aee2-0a0614c9a691 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:49.112782 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b3a617-7e66-4df7-aee2-0a0614c9a691 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:49.116544 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645937d2-aa9d-4b41-b77f-71980f485a1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:49.117146 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645937d2-aa9d-4b41-b77f-71980f485a1e service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:49.117421 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645937d2-aa9d-4b41-b77f-71980f485a1e service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:49.117686 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645937d2-aa9d-4b41-b77f-71980f485a1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:49.127056 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645937d2-aa9d-4b41-b77f-71980f485a1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:49.128514 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645937d2-aa9d-4b41-b77f-71980f485a1e service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:49.138040 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1125/2268] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:43:49 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:43:49.208232 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5312fbbb-186d-4d94-bac3-51427c5249a1 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:49.209994 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-5312fbbb-186d-4d94-bac3-51427c5249a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:49.216278 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645937d2-aa9d-4b41-b77f-71980f485a1e service cinder] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:49.225664 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645937d2-aa9d-4b41-b77f-71980f485a1e service cinder] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:49.234329 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8f993d-fd08-4f4f-ba13-c11e2af44be6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:49.234944 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f993d-fd08-4f4f-ba13-c11e2af44be6 service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:49.235244 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f993d-fd08-4f4f-ba13-c11e2af44be6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:49.235553 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f993d-fd08-4f4f-ba13-c11e2af44be6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:49.237481 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-4a8f993d-fd08-4f4f-ba13-c11e2af44be6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:49.302458 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5312fbbb-186d-4d94-bac3-51427c5249a1 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:49.307816 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2b75a-974d-4980-a164-bc5c0875460d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:49.308466 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2b75a-974d-4980-a164-bc5c0875460d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:49.308623 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8f993d-fd08-4f4f-ba13-c11e2af44be6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:49.308746 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2b75a-974d-4980-a164-bc5c0875460d admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:49.309560 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2b75a-974d-4980-a164-bc5c0875460d admin admin] PATH_INFO: `/v3/projects/38c645caf2e14858b9a117c04d1c7419` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:49.310602 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8f993d-fd08-4f4f-ba13-c11e2af44be6 service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:49.310744 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f2b75a-974d-4980-a164-bc5c0875460d admin admin] RBAC: Authorizing `identity:delete_project(project_id=38c645caf2e14858b9a117c04d1c7419)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:49.312712 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f2b75a-974d-4980-a164-bc5c0875460d admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:49.320428 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1126/2269] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:43:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:43:49.357717 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:49.365057 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-17f2b75a-974d-4980-a164-bc5c0875460d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38c645caf2e14858b9a117c04d1c7419', 'request_id': 'req-17f2b75a-974d-4980-a164-bc5c0875460d'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:49.373126 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-17f2b75a-974d-4980-a164-bc5c0875460d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38c645caf2e14858b9a117c04d1c7419', 'request_id': 'req-17f2b75a-974d-4980-a164-bc5c0875460d'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:49.381299 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-17f2b75a-974d-4980-a164-bc5c0875460d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38c645caf2e14858b9a117c04d1c7419', 'request_id': 'req-17f2b75a-974d-4980-a164-bc5c0875460d'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:49.396015 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-17f2b75a-974d-4980-a164-bc5c0875460d admin admin] The token cache is being invalidate because project 38c645caf2e14858b9a117c04d1c7419 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:43:49.396862 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-17f2b75a-974d-4980-a164-bc5c0875460d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:49.397366 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-17f2b75a-974d-4980-a164-bc5c0875460d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:49.398760 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1144/2270] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:43:49 2022] DELETE /identity/v3/projects/38c645caf2e14858b9a117c04d1c7419 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:49.404524 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8f993d-fd08-4f4f-ba13-c11e2af44be6 service cinder] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:49.417186 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8f993d-fd08-4f4f-ba13-c11e2af44be6 service cinder] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:49.491581 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3fa23f-5be4-4011-9a58-df4b1c5141aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:49.492213 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3fa23f-5be4-4011-9a58-df4b1c5141aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:49.492506 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3fa23f-5be4-4011-9a58-df4b1c5141aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:49.492757 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3fa23f-5be4-4011-9a58-df4b1c5141aa admin admin] PATH_INFO: `/v3/users/971e94bab5e74fbca9786ebba4f09d61` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:49.493896 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3fa23f-5be4-4011-9a58-df4b1c5141aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=971e94bab5e74fbca9786ebba4f09d61)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:49.495874 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3fa23f-5be4-4011-9a58-df4b1c5141aa admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:49.526044 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:49.548043 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-ef3fa23f-5be4-4011-9a58-df4b1c5141aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '971e94bab5e74fbca9786ebba4f09d61'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:49.548712 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-ef3fa23f-5be4-4011-9a58-df4b1c5141aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '971e94bab5e74fbca9786ebba4f09d61'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:49.562221 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-ef3fa23f-5be4-4011-9a58-df4b1c5141aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '971e94bab5e74fbca9786ebba4f09d61'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:49.570153 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-ef3fa23f-5be4-4011-9a58-df4b1c5141aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '971e94bab5e74fbca9786ebba4f09d61'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:49.577868 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-ef3fa23f-5be4-4011-9a58-df4b1c5141aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '971e94bab5e74fbca9786ebba4f09d61'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:49.584378 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1127/2271] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:43:49 2022] DELETE /identity/v3/users/971e94bab5e74fbca9786ebba4f09d61 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:49.850531 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2b75a-974d-4980-a164-bc5c0875460d admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:49.852087 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-17f2b75a-974d-4980-a164-bc5c0875460d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:49.867078 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2b75a-974d-4980-a164-bc5c0875460d admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:49.925896 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270805ac-b062-46f4-b8e3-2623d97d78ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:49.926534 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270805ac-b062-46f4-b8e3-2623d97d78ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:49.926818 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270805ac-b062-46f4-b8e3-2623d97d78ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:49.927094 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270805ac-b062-46f4-b8e3-2623d97d78ec admin admin] PATH_INFO: `/v3/projects/c86908735f7343ce90302e36ba62b9e5` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:49.928263 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270805ac-b062-46f4-b8e3-2623d97d78ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=c86908735f7343ce90302e36ba62b9e5)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:49.930168 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270805ac-b062-46f4-b8e3-2623d97d78ec admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:49.977293 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-270805ac-b062-46f4-b8e3-2623d97d78ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c86908735f7343ce90302e36ba62b9e5', 'request_id': 'req-270805ac-b062-46f4-b8e3-2623d97d78ec'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:49.983492 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-270805ac-b062-46f4-b8e3-2623d97d78ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c86908735f7343ce90302e36ba62b9e5', 'request_id': 'req-270805ac-b062-46f4-b8e3-2623d97d78ec'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:49.989558 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-270805ac-b062-46f4-b8e3-2623d97d78ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c86908735f7343ce90302e36ba62b9e5', 'request_id': 'req-270805ac-b062-46f4-b8e3-2623d97d78ec'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:50.002753 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-270805ac-b062-46f4-b8e3-2623d97d78ec admin admin] The token cache is being invalidate because project c86908735f7343ce90302e36ba62b9e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:43:50.003138 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-270805ac-b062-46f4-b8e3-2623d97d78ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:50.003903 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-270805ac-b062-46f4-b8e3-2623d97d78ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:50.005615 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1145/2272] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:43:49 2022] DELETE /identity/v3/projects/c86908735f7343ce90302e36ba62b9e5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:50.010808 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc6e83-2d52-4bd0-a665-69f1aa2d2765 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:50.011294 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc6e83-2d52-4bd0-a665-69f1aa2d2765 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:50.011428 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc6e83-2d52-4bd0-a665-69f1aa2d2765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:50.031429 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-96cc6e83-2d52-4bd0-a665-69f1aa2d2765 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:50.032620 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-96cc6e83-2d52-4bd0-a665-69f1aa2d2765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:50.091357 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1128/2273] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:50.100256 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530b63b-ac54-46f3-bb44-4a8e73b86561 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:50.100470 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530b63b-ac54-46f3-bb44-4a8e73b86561 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:50.100742 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530b63b-ac54-46f3-bb44-4a8e73b86561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:50.122288 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-9530b63b-ac54-46f3-bb44-4a8e73b86561 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:50.123514 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-9530b63b-ac54-46f3-bb44-4a8e73b86561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:50.131408 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1146/2274] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:50.135545 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc6e83-2d52-4bd0-a665-69f1aa2d2765 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:50.145752 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc6e83-2d52-4bd0-a665-69f1aa2d2765 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:50.150311 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a0dc57-8620-4f46-82bd-20ff765968dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:50.150902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0dc57-8620-4f46-82bd-20ff765968dd admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:50.151183 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0dc57-8620-4f46-82bd-20ff765968dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:50.151486 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0dc57-8620-4f46-82bd-20ff765968dd admin admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:50.152131 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a0dc57-8620-4f46-82bd-20ff765968dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:43:50.152434 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a0dc57-8620-4f46-82bd-20ff765968dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:50.154672 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a0dc57-8620-4f46-82bd-20ff765968dd admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:50.166339 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1129/2275] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 14:43:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:43:50.170809 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9530b63b-ac54-46f3-bb44-4a8e73b86561 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:50.180659 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9530b63b-ac54-46f3-bb44-4a8e73b86561 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:50.185497 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0260a7a-abfa-4ebc-8baa-6ad11c810142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:50.186089 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0260a7a-abfa-4ebc-8baa-6ad11c810142 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:50.186370 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0260a7a-abfa-4ebc-8baa-6ad11c810142 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:50.186643 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0260a7a-abfa-4ebc-8baa-6ad11c810142 admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:50.187326 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0260a7a-abfa-4ebc-8baa-6ad11c810142 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:50.189246 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0260a7a-abfa-4ebc-8baa-6ad11c810142 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:50.218357 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1147/2276] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:43:50 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:50.222409 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a0dc57-8620-4f46-82bd-20ff765968dd admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:50.232555 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a0dc57-8620-4f46-82bd-20ff765968dd admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:50.290410 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6dbd00-8a8b-4aca-a4ee-7ee45bb0f575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:50.291020 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6dbd00-8a8b-4aca-a4ee-7ee45bb0f575 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:50.291342 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6dbd00-8a8b-4aca-a4ee-7ee45bb0f575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:50.291618 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6dbd00-8a8b-4aca-a4ee-7ee45bb0f575 admin admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:50.292271 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6dbd00-8a8b-4aca-a4ee-7ee45bb0f575 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:50.294340 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6dbd00-8a8b-4aca-a4ee-7ee45bb0f575 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:50.316344 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1130/2277] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:43:50 2022] POST /identity/v3/users => generated 284 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:43:50.321812 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0260a7a-abfa-4ebc-8baa-6ad11c810142 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:50.333215 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0260a7a-abfa-4ebc-8baa-6ad11c810142 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:50.337988 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f4dcf5-0629-4b81-ab71-5a610cd4e072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:50.338592 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f4dcf5-0629-4b81-ab71-5a610cd4e072 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:50.338873 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f4dcf5-0629-4b81-ab71-5a610cd4e072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:50.339146 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f4dcf5-0629-4b81-ab71-5a610cd4e072 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:50.339825 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f4dcf5-0629-4b81-ab71-5a610cd4e072 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:50.342168 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f4dcf5-0629-4b81-ab71-5a610cd4e072 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:50.352880 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1148/2278] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:43:50.356981 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6dbd00-8a8b-4aca-a4ee-7ee45bb0f575 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:50.367729 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6dbd00-8a8b-4aca-a4ee-7ee45bb0f575 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:50.372472 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9b001e-8ffc-4510-b5d7-89d47c3141dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:50.373071 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b001e-8ffc-4510-b5d7-89d47c3141dc admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:50.373347 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b001e-8ffc-4510-b5d7-89d47c3141dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:50.373618 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b001e-8ffc-4510-b5d7-89d47c3141dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:50.374257 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9b001e-8ffc-4510-b5d7-89d47c3141dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:50.376330 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9b001e-8ffc-4510-b5d7-89d47c3141dc admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:50.387938 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1131/2279] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:43:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:43:50.393003 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f4dcf5-0629-4b81-ab71-5a610cd4e072 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:50.407067 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f4dcf5-0629-4b81-ab71-5a610cd4e072 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:50.415102 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f195e12-072d-4daf-9169-52731523f985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:50.415823 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f195e12-072d-4daf-9169-52731523f985 admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:50.416186 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f195e12-072d-4daf-9169-52731523f985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:50.416365 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f195e12-072d-4daf-9169-52731523f985 admin admin] PATH_INFO: `/v3/projects/a49798c20d924459b84ff50b8a44ff08/users/30292db03974498fb3de914e20ba7d09/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:50.434501 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f195e12-072d-4daf-9169-52731523f985 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a49798c20d924459b84ff50b8a44ff08, user_id=30292db03974498fb3de914e20ba7d09, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:50.437235 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f195e12-072d-4daf-9169-52731523f985 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:50.440859 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:50.454197 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1149/2280] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:43:50 2022] PUT /identity/v3/projects/a49798c20d924459b84ff50b8a44ff08/users/30292db03974498fb3de914e20ba7d09/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:50.464511 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10edd99-760d-4717-b171-8574ee8e6d44 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:50.464825 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10edd99-760d-4717-b171-8574ee8e6d44 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:50.465100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10edd99-760d-4717-b171-8574ee8e6d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:50.487751 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:50.500764 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-c10edd99-760d-4717-b171-8574ee8e6d44 None None] MFA Rules not processed for user `30292db03974498fb3de914e20ba7d09`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:50.502382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-c10edd99-760d-4717-b171-8574ee8e6d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:50.569827 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1132/2281] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:50.582024 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f195e12-072d-4daf-9169-52731523f985 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:50.583758 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-9f195e12-072d-4daf-9169-52731523f985 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:50.682396 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f195e12-072d-4daf-9169-52731523f985 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:50.687952 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df374ba7-b9fb-473b-a72a-c3c7d44a0815 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6889402e54424bbd8d42c7bb759cf700', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:50.688526 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df374ba7-b9fb-473b-a72a-c3c7d44a0815 service glance] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:50.688943 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df374ba7-b9fb-473b-a72a-c3c7d44a0815 service glance] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:50.689157 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df374ba7-b9fb-473b-a72a-c3c7d44a0815 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:50.698900 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df374ba7-b9fb-473b-a72a-c3c7d44a0815 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:50.701699 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df374ba7-b9fb-473b-a72a-c3c7d44a0815 service glance] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:50.713317 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1150/2282] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:43:50 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:43:55.883053 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10edd99-760d-4717-b171-8574ee8e6d44 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:55.884536 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-c10edd99-760d-4717-b171-8574ee8e6d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:55.949138 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10edd99-760d-4717-b171-8574ee8e6d44 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:55.954448 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9209ab-ac15-492d-9d88-e87935f62e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:55.955100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9209ab-ac15-492d-9d88-e87935f62e43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:55.955561 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9209ab-ac15-492d-9d88-e87935f62e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:55.955712 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9209ab-ac15-492d-9d88-e87935f62e43 admin admin] PATH_INFO: `/v3/users/f254345518c44cc295233adfec37b93c` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:55.956829 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9209ab-ac15-492d-9d88-e87935f62e43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f254345518c44cc295233adfec37b93c)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:55.958743 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9209ab-ac15-492d-9d88-e87935f62e43 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:56.004316 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-bb9209ab-ac15-492d-9d88-e87935f62e43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f254345518c44cc295233adfec37b93c'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:56.004984 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-bb9209ab-ac15-492d-9d88-e87935f62e43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f254345518c44cc295233adfec37b93c'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:56.017320 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-bb9209ab-ac15-492d-9d88-e87935f62e43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f254345518c44cc295233adfec37b93c'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:56.024033 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-bb9209ab-ac15-492d-9d88-e87935f62e43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f254345518c44cc295233adfec37b93c'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:56.031357 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-bb9209ab-ac15-492d-9d88-e87935f62e43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f254345518c44cc295233adfec37b93c'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:56.038608 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1133/2283] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:43:55 2022] DELETE /identity/v3/users/f254345518c44cc295233adfec37b93c => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:43:56.047652 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f333cd-b78e-4446-9ecb-b9447f87c877 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:56.048144 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f333cd-b78e-4446-9ecb-b9447f87c877 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:56.048233 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f333cd-b78e-4446-9ecb-b9447f87c877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:56.068590 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-e3f333cd-b78e-4446-9ecb-b9447f87c877 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:43:56.070036 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-e3f333cd-b78e-4446-9ecb-b9447f87c877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:56.145897 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1151/2284] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:43:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:43:56.155814 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9209ab-ac15-492d-9d88-e87935f62e43 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:56.157315 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-bb9209ab-ac15-492d-9d88-e87935f62e43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:56.179527 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:43:56.216105 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9209ab-ac15-492d-9d88-e87935f62e43 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:56.220015 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e55facf-75c6-4f38-8cde-04015a31bbd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:56.220634 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e55facf-75c6-4f38-8cde-04015a31bbd2 service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:56.220917 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e55facf-75c6-4f38-8cde-04015a31bbd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:56.221197 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e55facf-75c6-4f38-8cde-04015a31bbd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:56.231313 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e55facf-75c6-4f38-8cde-04015a31bbd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:56.232748 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e55facf-75c6-4f38-8cde-04015a31bbd2 service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:56.242308 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1134/2285] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:43:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:43:56.513317 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f333cd-b78e-4446-9ecb-b9447f87c877 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:43:56.515181 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-e3f333cd-b78e-4446-9ecb-b9447f87c877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:43:56.542254 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f333cd-b78e-4446-9ecb-b9447f87c877 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:43:56.549775 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e1bd3-0d08-4b4b-9482-5e4237404d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:43:56.550813 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e1bd3-0d08-4b4b-9482-5e4237404d83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:43:56.551192 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e1bd3-0d08-4b4b-9482-5e4237404d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:43:56.551593 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e1bd3-0d08-4b4b-9482-5e4237404d83 admin admin] PATH_INFO: `/v3/projects/bdbc620c36b34e79ab20f16a08721b89` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:43:56.552954 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15e1bd3-0d08-4b4b-9482-5e4237404d83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdbc620c36b34e79ab20f16a08721b89)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:43:56.555971 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15e1bd3-0d08-4b4b-9482-5e4237404d83 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:43:56.649064 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-d15e1bd3-0d08-4b4b-9482-5e4237404d83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdbc620c36b34e79ab20f16a08721b89', 'request_id': 'req-d15e1bd3-0d08-4b4b-9482-5e4237404d83'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:56.661163 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-d15e1bd3-0d08-4b4b-9482-5e4237404d83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdbc620c36b34e79ab20f16a08721b89', 'request_id': 'req-d15e1bd3-0d08-4b4b-9482-5e4237404d83'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:56.668248 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-d15e1bd3-0d08-4b4b-9482-5e4237404d83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdbc620c36b34e79ab20f16a08721b89', 'request_id': 'req-d15e1bd3-0d08-4b4b-9482-5e4237404d83'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:56.685624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-d15e1bd3-0d08-4b4b-9482-5e4237404d83 admin admin] The token cache is being invalidate because project bdbc620c36b34e79ab20f16a08721b89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:43:56.686043 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-d15e1bd3-0d08-4b4b-9482-5e4237404d83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:56.686760 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-d15e1bd3-0d08-4b4b-9482-5e4237404d83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:43:56.699284 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1152/2286] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:43:56 2022] DELETE /identity/v3/projects/bdbc620c36b34e79ab20f16a08721b89 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:06.237975 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=17 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:06.681622 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:34.977301 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e55facf-75c6-4f38-8cde-04015a31bbd2 service neutron] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:34.978837 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-2e55facf-75c6-4f38-8cde-04015a31bbd2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:34.993026 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e55facf-75c6-4f38-8cde-04015a31bbd2 service neutron] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:35.052649 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c36eb39-ac9d-4d57-9413-c4579426dc50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:35.053314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c36eb39-ac9d-4d57-9413-c4579426dc50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:35.053593 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c36eb39-ac9d-4d57-9413-c4579426dc50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:35.053885 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c36eb39-ac9d-4d57-9413-c4579426dc50 admin admin] PATH_INFO: `/v3/users/7839cf561cda46e1a32100e697b71e09` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:35.067407 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c36eb39-ac9d-4d57-9413-c4579426dc50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7839cf561cda46e1a32100e697b71e09)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:35.067407 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c36eb39-ac9d-4d57-9413-c4579426dc50 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:35.106569 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-0c36eb39-ac9d-4d57-9413-c4579426dc50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7839cf561cda46e1a32100e697b71e09'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:35.107248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-0c36eb39-ac9d-4d57-9413-c4579426dc50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7839cf561cda46e1a32100e697b71e09'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:35.119806 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-0c36eb39-ac9d-4d57-9413-c4579426dc50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7839cf561cda46e1a32100e697b71e09'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:35.127149 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-0c36eb39-ac9d-4d57-9413-c4579426dc50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7839cf561cda46e1a32100e697b71e09'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:35.134897 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-0c36eb39-ac9d-4d57-9413-c4579426dc50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7839cf561cda46e1a32100e697b71e09'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:35.141849 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1135/2287] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:44:34 2022] DELETE /identity/v3/users/7839cf561cda46e1a32100e697b71e09 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:35.146338 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f024be1-db42-4a07-8726-47a805e92257 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:35.146837 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f024be1-db42-4a07-8726-47a805e92257 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:35.146933 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f024be1-db42-4a07-8726-47a805e92257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:35.166945 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-6f024be1-db42-4a07-8726-47a805e92257 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:35.168494 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-6f024be1-db42-4a07-8726-47a805e92257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:35.257641 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1153/2288] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:44:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:35.268297 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c36eb39-ac9d-4d57-9413-c4579426dc50 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:35.269723 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-0c36eb39-ac9d-4d57-9413-c4579426dc50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:35.361017 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c36eb39-ac9d-4d57-9413-c4579426dc50 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:35.365450 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526ac7d4-01e6-4eea-a5f8-4fe91be81caa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:35.366089 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ac7d4-01e6-4eea-a5f8-4fe91be81caa service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:35.366366 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ac7d4-01e6-4eea-a5f8-4fe91be81caa service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:35.366637 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ac7d4-01e6-4eea-a5f8-4fe91be81caa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:35.381618 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526ac7d4-01e6-4eea-a5f8-4fe91be81caa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:35.384918 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526ac7d4-01e6-4eea-a5f8-4fe91be81caa service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:35.397982 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1136/2289] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:44:35.766527 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f024be1-db42-4a07-8726-47a805e92257 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:35.768467 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-6f024be1-db42-4a07-8726-47a805e92257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:35.779490 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526ac7d4-01e6-4eea-a5f8-4fe91be81caa service neutron] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:35.781241 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-526ac7d4-01e6-4eea-a5f8-4fe91be81caa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:35.781575 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f024be1-db42-4a07-8726-47a805e92257 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:35.786831 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e67f680-b2f0-4526-afa5-14a940906c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:35.787494 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e67f680-b2f0-4526-afa5-14a940906c66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:35.788227 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e67f680-b2f0-4526-afa5-14a940906c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:35.788662 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e67f680-b2f0-4526-afa5-14a940906c66 admin admin] PATH_INFO: `/v3/projects/9375fb38ac7a40c1aba16d4185ae0dd5` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:35.789626 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e67f680-b2f0-4526-afa5-14a940906c66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9375fb38ac7a40c1aba16d4185ae0dd5)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:35.791559 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e67f680-b2f0-4526-afa5-14a940906c66 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:35.807156 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526ac7d4-01e6-4eea-a5f8-4fe91be81caa service neutron] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:35.813337 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29076b60-bf12-4516-a97c-885d86d1bed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:35.813852 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29076b60-bf12-4516-a97c-885d86d1bed1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:35.814132 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29076b60-bf12-4516-a97c-885d86d1bed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:35.814404 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29076b60-bf12-4516-a97c-885d86d1bed1 admin admin] PATH_INFO: `/v3/users/b40ccb34a65e4c73820760918d7ddbd7` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:35.815563 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29076b60-bf12-4516-a97c-885d86d1bed1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b40ccb34a65e4c73820760918d7ddbd7)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:35.817734 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29076b60-bf12-4516-a97c-885d86d1bed1 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:35.837735 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:35.843549 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-5e67f680-b2f0-4526-afa5-14a940906c66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9375fb38ac7a40c1aba16d4185ae0dd5', 'request_id': 'req-5e67f680-b2f0-4526-afa5-14a940906c66'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:35.849444 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-5e67f680-b2f0-4526-afa5-14a940906c66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9375fb38ac7a40c1aba16d4185ae0dd5', 'request_id': 'req-5e67f680-b2f0-4526-afa5-14a940906c66'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:35.856777 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-5e67f680-b2f0-4526-afa5-14a940906c66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9375fb38ac7a40c1aba16d4185ae0dd5', 'request_id': 'req-5e67f680-b2f0-4526-afa5-14a940906c66'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:35.869233 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-5e67f680-b2f0-4526-afa5-14a940906c66 admin admin] The token cache is being invalidate because project 9375fb38ac7a40c1aba16d4185ae0dd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:44:35.869524 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-5e67f680-b2f0-4526-afa5-14a940906c66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:35.870772 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-5e67f680-b2f0-4526-afa5-14a940906c66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:35.872316 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1154/2290] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:44:35 2022] DELETE /identity/v3/projects/9375fb38ac7a40c1aba16d4185ae0dd5 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:35.875735 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-29076b60-bf12-4516-a97c-885d86d1bed1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b40ccb34a65e4c73820760918d7ddbd7'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:35.875945 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-29076b60-bf12-4516-a97c-885d86d1bed1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b40ccb34a65e4c73820760918d7ddbd7'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:35.876959 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eedae29-48ef-4ce4-bbea-6c22aee97f1a None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:35.877241 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eedae29-48ef-4ce4-bbea-6c22aee97f1a None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:35.877513 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eedae29-48ef-4ce4-bbea-6c22aee97f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:35.889361 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-29076b60-bf12-4516-a97c-885d86d1bed1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b40ccb34a65e4c73820760918d7ddbd7'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:35.896678 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-29076b60-bf12-4516-a97c-885d86d1bed1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b40ccb34a65e4c73820760918d7ddbd7'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:35.896987 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-6eedae29-48ef-4ce4-bbea-6c22aee97f1a None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:35.898197 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-6eedae29-48ef-4ce4-bbea-6c22aee97f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:35.906384 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-29076b60-bf12-4516-a97c-885d86d1bed1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b40ccb34a65e4c73820760918d7ddbd7'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:35.913284 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1137/2291] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:44:35 2022] DELETE /identity/v3/users/b40ccb34a65e4c73820760918d7ddbd7 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:44:35.918001 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a3b67-3a50-41fe-ab0a-8a17ff126b24 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:35.918291 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a3b67-3a50-41fe-ab0a-8a17ff126b24 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:35.918569 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a3b67-3a50-41fe-ab0a-8a17ff126b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:35.937967 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-389a3b67-3a50-41fe-ab0a-8a17ff126b24 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:35.939283 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-389a3b67-3a50-41fe-ab0a-8a17ff126b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:35.960868 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1155/2292] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:44:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:44:35.969697 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35fb018-f633-4c62-b841-5342ac0aeafd None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:35.969961 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35fb018-f633-4c62-b841-5342ac0aeafd None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:35.970243 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35fb018-f633-4c62-b841-5342ac0aeafd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:35.991082 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-d35fb018-f633-4c62-b841-5342ac0aeafd None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:35.992329 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-d35fb018-f633-4c62-b841-5342ac0aeafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:36.001354 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1138/2293] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:44:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:44:36.012318 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389a3b67-3a50-41fe-ab0a-8a17ff126b24 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:36.013478 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:36.014471 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-389a3b67-3a50-41fe-ab0a-8a17ff126b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:36.057065 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1156/2294] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:44:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:44:36.062028 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35fb018-f633-4c62-b841-5342ac0aeafd None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:36.071675 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35fb018-f633-4c62-b841-5342ac0aeafd None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:36.076567 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389a3b67-3a50-41fe-ab0a-8a17ff126b24 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:36.076700 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ef538-62e2-4b89-9b0f-9a1bbd09bf33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:36.077281 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ef538-62e2-4b89-9b0f-9a1bbd09bf33 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:36.077563 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ef538-62e2-4b89-9b0f-9a1bbd09bf33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:36.077864 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ef538-62e2-4b89-9b0f-9a1bbd09bf33 admin admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:36.078466 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2ef538-62e2-4b89-9b0f-9a1bbd09bf33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:44:36.078767 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2ef538-62e2-4b89-9b0f-9a1bbd09bf33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:36.080391 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2ef538-62e2-4b89-9b0f-9a1bbd09bf33 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:36.080504 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1045ebb6-50bb-4556-b310-32385395ab6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:36.081104 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1045ebb6-50bb-4556-b310-32385395ab6b service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:36.081382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1045ebb6-50bb-4556-b310-32385395ab6b service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:36.081652 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1045ebb6-50bb-4556-b310-32385395ab6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:36.093730 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1045ebb6-50bb-4556-b310-32385395ab6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:36.095180 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1045ebb6-50bb-4556-b310-32385395ab6b service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:36.095771 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1157/2295] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 14:44:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 14:44:36.099890 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ef538-62e2-4b89-9b0f-9a1bbd09bf33 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:36.109877 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1139/2296] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:44:36.111444 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ef538-62e2-4b89-9b0f-9a1bbd09bf33 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:36.116993 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8722dd-93ec-43a0-a0a0-b9c4c9329e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:36.117400 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8722dd-93ec-43a0-a0a0-b9c4c9329e14 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:36.117687 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8722dd-93ec-43a0-a0a0-b9c4c9329e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:36.118292 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8722dd-93ec-43a0-a0a0-b9c4c9329e14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:36.118806 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8722dd-93ec-43a0-a0a0-b9c4c9329e14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:36.121513 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8722dd-93ec-43a0-a0a0-b9c4c9329e14 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:36.151560 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1158/2297] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:44:36 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:44:36.156415 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1045ebb6-50bb-4556-b310-32385395ab6b service neutron] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:36.171349 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1045ebb6-50bb-4556-b310-32385395ab6b service neutron] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:36.223260 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:36.256713 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1e082-8d1a-4ed5-b096-4e22b5406993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:36.257304 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1e082-8d1a-4ed5-b096-4e22b5406993 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:36.257620 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1e082-8d1a-4ed5-b096-4e22b5406993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:36.257895 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1e082-8d1a-4ed5-b096-4e22b5406993 admin admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:36.258532 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f1e082-8d1a-4ed5-b096-4e22b5406993 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:36.260617 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f1e082-8d1a-4ed5-b096-4e22b5406993 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:36.288902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1140/2298] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:44:36 2022] POST /identity/v3/users => generated 287 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:44:36.290190 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8722dd-93ec-43a0-a0a0-b9c4c9329e14 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:36.300742 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8722dd-93ec-43a0-a0a0-b9c4c9329e14 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:36.305659 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b450b8-c564-44e1-b3b1-216ef462b311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:36.306372 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b450b8-c564-44e1-b3b1-216ef462b311 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:36.306659 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b450b8-c564-44e1-b3b1-216ef462b311 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:36.306957 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b450b8-c564-44e1-b3b1-216ef462b311 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:36.308249 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b450b8-c564-44e1-b3b1-216ef462b311 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:36.309847 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b450b8-c564-44e1-b3b1-216ef462b311 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:36.320488 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1159/2299] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:44:36.325367 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1e082-8d1a-4ed5-b096-4e22b5406993 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:36.335885 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1e082-8d1a-4ed5-b096-4e22b5406993 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:36.341465 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd88774-5949-4cee-858d-a455d3ea9d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:36.342070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd88774-5949-4cee-858d-a455d3ea9d2e admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:36.342353 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd88774-5949-4cee-858d-a455d3ea9d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:36.342633 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd88774-5949-4cee-858d-a455d3ea9d2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:36.343298 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd88774-5949-4cee-858d-a455d3ea9d2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:36.344897 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd88774-5949-4cee-858d-a455d3ea9d2e admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:36.358365 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1141/2300] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:44:36.359669 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b450b8-c564-44e1-b3b1-216ef462b311 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:36.369565 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b450b8-c564-44e1-b3b1-216ef462b311 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:36.375567 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca775006-d078-4aa0-9c9d-d1dc1ad1575f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:36.376300 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca775006-d078-4aa0-9c9d-d1dc1ad1575f admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:36.376667 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca775006-d078-4aa0-9c9d-d1dc1ad1575f admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:36.377002 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca775006-d078-4aa0-9c9d-d1dc1ad1575f admin admin] PATH_INFO: `/v3/projects/fe47f61a4dbf4e669ff257d299e92ee4/users/08b4f8865dd14290ab714039a2e424fb/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:36.393966 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca775006-d078-4aa0-9c9d-d1dc1ad1575f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe47f61a4dbf4e669ff257d299e92ee4, user_id=08b4f8865dd14290ab714039a2e424fb, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:36.396695 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca775006-d078-4aa0-9c9d-d1dc1ad1575f admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:36.407288 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd88774-5949-4cee-858d-a455d3ea9d2e admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:36.409872 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-8bd88774-5949-4cee-858d-a455d3ea9d2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:36.410707 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1160/2301] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:44:36 2022] PUT /identity/v3/projects/fe47f61a4dbf4e669ff257d299e92ee4/users/08b4f8865dd14290ab714039a2e424fb/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:36.422379 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d087e4e-bc49-4651-88a3-ce9c1911a396 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:36.422665 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d087e4e-bc49-4651-88a3-ce9c1911a396 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:36.422937 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d087e4e-bc49-4651-88a3-ce9c1911a396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:36.425840 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd88774-5949-4cee-858d-a455d3ea9d2e admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:36.453128 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-6d087e4e-bc49-4651-88a3-ce9c1911a396 None None] MFA Rules not processed for user `08b4f8865dd14290ab714039a2e424fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:36.454348 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-6d087e4e-bc49-4651-88a3-ce9c1911a396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:36.487838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caba20e-b66b-4e30-8b95-72ebb14ac690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:36.488481 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caba20e-b66b-4e30-8b95-72ebb14ac690 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:36.488766 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caba20e-b66b-4e30-8b95-72ebb14ac690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:36.489042 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caba20e-b66b-4e30-8b95-72ebb14ac690 admin admin] PATH_INFO: `/v3/projects/94413b61a4d34324be36806a1a09ede3` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:36.490437 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caba20e-b66b-4e30-8b95-72ebb14ac690 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94413b61a4d34324be36806a1a09ede3)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:36.492199 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caba20e-b66b-4e30-8b95-72ebb14ac690 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:36.534719 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1161/2302] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:44:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:36.542807 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d087e4e-bc49-4651-88a3-ce9c1911a396 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:36.545296 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-6d087e4e-bc49-4651-88a3-ce9c1911a396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:36.552916 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-5caba20e-b66b-4e30-8b95-72ebb14ac690 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94413b61a4d34324be36806a1a09ede3', 'request_id': 'req-5caba20e-b66b-4e30-8b95-72ebb14ac690'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:36.560451 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-5caba20e-b66b-4e30-8b95-72ebb14ac690 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94413b61a4d34324be36806a1a09ede3', 'request_id': 'req-5caba20e-b66b-4e30-8b95-72ebb14ac690'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:36.568194 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-5caba20e-b66b-4e30-8b95-72ebb14ac690 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94413b61a4d34324be36806a1a09ede3', 'request_id': 'req-5caba20e-b66b-4e30-8b95-72ebb14ac690'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:36.600941 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-5caba20e-b66b-4e30-8b95-72ebb14ac690 admin admin] The token cache is being invalidate because project 94413b61a4d34324be36806a1a09ede3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:44:36.601232 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-5caba20e-b66b-4e30-8b95-72ebb14ac690 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:36.603891 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-5caba20e-b66b-4e30-8b95-72ebb14ac690 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:36.606763 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1142/2303] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:44:36 2022] DELETE /identity/v3/projects/94413b61a4d34324be36806a1a09ede3 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:44:36.611420 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caba20e-b66b-4e30-8b95-72ebb14ac690 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:36.614111 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-5caba20e-b66b-4e30-8b95-72ebb14ac690 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:36.649761 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:36.668607 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d087e4e-bc49-4651-88a3-ce9c1911a396 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:36.704672 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:36.727742 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caba20e-b66b-4e30-8b95-72ebb14ac690 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:36.744410 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6695521c-6a75-4a9a-9956-c9f17f1b94eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:36.746355 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fd5ffb-4793-4bc7-8cf7-c40e7027056e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:36.747007 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd5ffb-4793-4bc7-8cf7-c40e7027056e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:36.747325 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd5ffb-4793-4bc7-8cf7-c40e7027056e admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:36.747603 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd5ffb-4793-4bc7-8cf7-c40e7027056e admin admin] PATH_INFO: `/v3/users/691e3af987ba4971a7af24efa11c75d2` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:36.748295 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695521c-6a75-4a9a-9956-c9f17f1b94eb service cinder] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:36.748971 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695521c-6a75-4a9a-9956-c9f17f1b94eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:36.748971 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695521c-6a75-4a9a-9956-c9f17f1b94eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:36.750024 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fd5ffb-4793-4bc7-8cf7-c40e7027056e admin admin] RBAC: Authorizing `identity:delete_user(user_id=691e3af987ba4971a7af24efa11c75d2)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:36.751999 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fd5ffb-4793-4bc7-8cf7-c40e7027056e admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:36.755722 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-6695521c-6a75-4a9a-9956-c9f17f1b94eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:36.827993 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-a7fd5ffb-4793-4bc7-8cf7-c40e7027056e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '691e3af987ba4971a7af24efa11c75d2'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:36.829478 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-a7fd5ffb-4793-4bc7-8cf7-c40e7027056e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '691e3af987ba4971a7af24efa11c75d2'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:36.848043 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6695521c-6a75-4a9a-9956-c9f17f1b94eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:36.848903 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-a7fd5ffb-4793-4bc7-8cf7-c40e7027056e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '691e3af987ba4971a7af24efa11c75d2'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:36.850809 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6695521c-6a75-4a9a-9956-c9f17f1b94eb service cinder] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:36.857347 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-a7fd5ffb-4793-4bc7-8cf7-c40e7027056e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '691e3af987ba4971a7af24efa11c75d2'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:36.861392 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1162/2304] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:44:36.871775 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-a7fd5ffb-4793-4bc7-8cf7-c40e7027056e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '691e3af987ba4971a7af24efa11c75d2'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:36.878557 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1143/2305] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:44:36 2022] DELETE /identity/v3/users/691e3af987ba4971a7af24efa11c75d2 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:37.225744 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6695521c-6a75-4a9a-9956-c9f17f1b94eb service cinder] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:37.249634 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6695521c-6a75-4a9a-9956-c9f17f1b94eb service cinder] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:37.317476 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891b850a-68b5-4ff2-bc40-add23f19ab5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:37.318247 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891b850a-68b5-4ff2-bc40-add23f19ab5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:37.318534 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891b850a-68b5-4ff2-bc40-add23f19ab5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:37.318801 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891b850a-68b5-4ff2-bc40-add23f19ab5c admin admin] PATH_INFO: `/v3/projects/894adc4e436f49f58315b9923c5a2563` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:37.319972 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891b850a-68b5-4ff2-bc40-add23f19ab5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=894adc4e436f49f58315b9923c5a2563)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:37.321892 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891b850a-68b5-4ff2-bc40-add23f19ab5c admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:37.374493 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-891b850a-68b5-4ff2-bc40-add23f19ab5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '894adc4e436f49f58315b9923c5a2563', 'request_id': 'req-891b850a-68b5-4ff2-bc40-add23f19ab5c'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:37.379054 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:37.380340 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-891b850a-68b5-4ff2-bc40-add23f19ab5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '894adc4e436f49f58315b9923c5a2563', 'request_id': 'req-891b850a-68b5-4ff2-bc40-add23f19ab5c'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:37.388874 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-891b850a-68b5-4ff2-bc40-add23f19ab5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '894adc4e436f49f58315b9923c5a2563', 'request_id': 'req-891b850a-68b5-4ff2-bc40-add23f19ab5c'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:37.402897 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-891b850a-68b5-4ff2-bc40-add23f19ab5c admin admin] The token cache is being invalidate because project 894adc4e436f49f58315b9923c5a2563 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:44:37.403191 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-891b850a-68b5-4ff2-bc40-add23f19ab5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:37.403864 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-891b850a-68b5-4ff2-bc40-add23f19ab5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:37.405356 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1163/2306] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:44:37 2022] DELETE /identity/v3/projects/894adc4e436f49f58315b9923c5a2563 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:37.410926 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9777788-323c-4a5b-b6ad-cb3e1b160a56 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:37.411619 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9777788-323c-4a5b-b6ad-cb3e1b160a56 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:37.412119 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9777788-323c-4a5b-b6ad-cb3e1b160a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:37.432722 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-a9777788-323c-4a5b-b6ad-cb3e1b160a56 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:37.434020 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-a9777788-323c-4a5b-b6ad-cb3e1b160a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:37.495560 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1144/2307] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:44:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:37.503040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f1aac-edd4-4a1a-8699-d652f51241ab None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:37.503362 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f1aac-edd4-4a1a-8699-d652f51241ab None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:37.503630 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f1aac-edd4-4a1a-8699-d652f51241ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:37.524515 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-836f1aac-edd4-4a1a-8699-d652f51241ab None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:37.525733 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-836f1aac-edd4-4a1a-8699-d652f51241ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:37.534256 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1164/2308] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:44:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:37.538451 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9777788-323c-4a5b-b6ad-cb3e1b160a56 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:37.548349 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9777788-323c-4a5b-b6ad-cb3e1b160a56 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:37.552870 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d42a1e-f3fc-4f37-878b-6f6d24665c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:37.553462 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d42a1e-f3fc-4f37-878b-6f6d24665c66 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:37.553769 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d42a1e-f3fc-4f37-878b-6f6d24665c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:37.554039 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d42a1e-f3fc-4f37-878b-6f6d24665c66 admin admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:37.554684 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d42a1e-f3fc-4f37-878b-6f6d24665c66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:44:37.554982 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d42a1e-f3fc-4f37-878b-6f6d24665c66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:37.556611 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d42a1e-f3fc-4f37-878b-6f6d24665c66 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:37.568297 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1145/2309] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 14:44:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:44:37.572470 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836f1aac-edd4-4a1a-8699-d652f51241ab None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:37.582299 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836f1aac-edd4-4a1a-8699-d652f51241ab None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:37.586954 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b993ab-5bcb-4162-8aed-b6be85234be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:37.587585 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b993ab-5bcb-4162-8aed-b6be85234be6 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:37.587863 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b993ab-5bcb-4162-8aed-b6be85234be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:37.588130 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b993ab-5bcb-4162-8aed-b6be85234be6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:37.589361 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b993ab-5bcb-4162-8aed-b6be85234be6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:37.591244 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b993ab-5bcb-4162-8aed-b6be85234be6 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:37.617594 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1165/2310] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:44:37 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:44:37.621782 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d42a1e-f3fc-4f37-878b-6f6d24665c66 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:37.632087 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d42a1e-f3fc-4f37-878b-6f6d24665c66 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:37.679735 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:37.690340 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d826e4bf-c121-4eed-9ef4-a6367f83ab54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:37.691201 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826e4bf-c121-4eed-9ef4-a6367f83ab54 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:37.691506 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826e4bf-c121-4eed-9ef4-a6367f83ab54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:37.691779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826e4bf-c121-4eed-9ef4-a6367f83ab54 admin admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:37.692561 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d826e4bf-c121-4eed-9ef4-a6367f83ab54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:37.694290 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d826e4bf-c121-4eed-9ef4-a6367f83ab54 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:37.717383 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1146/2311] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:44:37 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:44:37.721437 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b993ab-5bcb-4162-8aed-b6be85234be6 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:37.731618 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b993ab-5bcb-4162-8aed-b6be85234be6 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:37.736343 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2635af3d-68ce-4c36-ac7f-094176bcd535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:37.736944 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635af3d-68ce-4c36-ac7f-094176bcd535 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:37.737221 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635af3d-68ce-4c36-ac7f-094176bcd535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:37.737492 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635af3d-68ce-4c36-ac7f-094176bcd535 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:37.738133 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2635af3d-68ce-4c36-ac7f-094176bcd535 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:37.739795 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2635af3d-68ce-4c36-ac7f-094176bcd535 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:37.752637 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1166/2312] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:44:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:44:37.756527 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d826e4bf-c121-4eed-9ef4-a6367f83ab54 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:37.774625 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d826e4bf-c121-4eed-9ef4-a6367f83ab54 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:37.779690 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f57257-0f5f-420b-8886-b4daef4340ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:37.780298 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f57257-0f5f-420b-8886-b4daef4340ef admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:37.780578 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f57257-0f5f-420b-8886-b4daef4340ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:37.780850 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f57257-0f5f-420b-8886-b4daef4340ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:37.781517 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f57257-0f5f-420b-8886-b4daef4340ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:37.783125 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f57257-0f5f-420b-8886-b4daef4340ef admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:37.797607 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1147/2313] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:44:37 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:44:37.798998 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2635af3d-68ce-4c36-ac7f-094176bcd535 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:37.810070 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2635af3d-68ce-4c36-ac7f-094176bcd535 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:37.815072 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6214bde6-c96e-466a-b01a-2a3d6fff8315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:37.816964 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214bde6-c96e-466a-b01a-2a3d6fff8315 admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:37.816964 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214bde6-c96e-466a-b01a-2a3d6fff8315 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:37.816964 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214bde6-c96e-466a-b01a-2a3d6fff8315 admin admin] PATH_INFO: `/v3/projects/fd26aaeccd554bb9923f4287cf9e06da/users/65539b4338a74f53af7d4813f6d6ac44/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:37.835359 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6214bde6-c96e-466a-b01a-2a3d6fff8315 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd26aaeccd554bb9923f4287cf9e06da, user_id=65539b4338a74f53af7d4813f6d6ac44, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:37.838108 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6214bde6-c96e-466a-b01a-2a3d6fff8315 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:37.858238 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1167/2314] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:44:37 2022] PUT /identity/v3/projects/fd26aaeccd554bb9923f4287cf9e06da/users/65539b4338a74f53af7d4813f6d6ac44/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:37.863136 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d896f0-c1b0-4e79-bad9-4c306f7de5ec None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:37.863453 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d896f0-c1b0-4e79-bad9-4c306f7de5ec None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:37.863729 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d896f0-c1b0-4e79-bad9-4c306f7de5ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:37.882461 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-b4d896f0-c1b0-4e79-bad9-4c306f7de5ec None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:37.884146 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-b4d896f0-c1b0-4e79-bad9-4c306f7de5ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:37.945086 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1148/2315] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:44:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:44:37.958662 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6214bde6-c96e-466a-b01a-2a3d6fff8315 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:37.960460 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-6214bde6-c96e-466a-b01a-2a3d6fff8315 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:38.034860 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6214bde6-c96e-466a-b01a-2a3d6fff8315 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:38.038390 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8870cab-a460-4881-8db1-7c2e5efc4727 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:38.039580 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8870cab-a460-4881-8db1-7c2e5efc4727 service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:38.039898 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8870cab-a460-4881-8db1-7c2e5efc4727 service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:38.040182 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8870cab-a460-4881-8db1-7c2e5efc4727 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:38.050635 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8870cab-a460-4881-8db1-7c2e5efc4727 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:38.052099 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8870cab-a460-4881-8db1-7c2e5efc4727 service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:38.061692 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1168/2316] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:44:38.747945 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae218de-c096-4f9c-a090-264401af6ebb None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:38.748398 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae218de-c096-4f9c-a090-264401af6ebb None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:38.748497 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae218de-c096-4f9c-a090-264401af6ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:38.777017 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-cae218de-c096-4f9c-a090-264401af6ebb None None] MFA Rules not processed for user `65539b4338a74f53af7d4813f6d6ac44`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:38.778280 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-cae218de-c096-4f9c-a090-264401af6ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:38.847033 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1149/2317] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:44:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:38.851426 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8870cab-a460-4881-8db1-7c2e5efc4727 service neutron] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:38.862299 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8870cab-a460-4881-8db1-7c2e5efc4727 service neutron] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:38.867143 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a83ec7-5304-408d-aac6-1211ee567169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:38.867987 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a83ec7-5304-408d-aac6-1211ee567169 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:38.869227 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a83ec7-5304-408d-aac6-1211ee567169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:38.869680 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a83ec7-5304-408d-aac6-1211ee567169 admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:38.870714 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a83ec7-5304-408d-aac6-1211ee567169 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:38.873094 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a83ec7-5304-408d-aac6-1211ee567169 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:38.910675 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1169/2318] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:44:38 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:44:38.914548 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae218de-c096-4f9c-a090-264401af6ebb None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:38.926126 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae218de-c096-4f9c-a090-264401af6ebb None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:38.987430 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a96c0-cabc-4a6c-b29f-e307c8922400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:38.988324 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a96c0-cabc-4a6c-b29f-e307c8922400 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:38.988856 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a96c0-cabc-4a6c-b29f-e307c8922400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:38.989127 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a96c0-cabc-4a6c-b29f-e307c8922400 admin admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:38.989762 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605a96c0-cabc-4a6c-b29f-e307c8922400 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:38.991742 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605a96c0-cabc-4a6c-b29f-e307c8922400 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:39.002098 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:39.019818 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1150/2319] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:44:38 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:44:39.020691 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a83ec7-5304-408d-aac6-1211ee567169 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:39.031761 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a83ec7-5304-408d-aac6-1211ee567169 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:39.036797 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c54ab17-1df8-4246-aebd-6df4e1147296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:39.037410 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54ab17-1df8-4246-aebd-6df4e1147296 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:39.037734 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54ab17-1df8-4246-aebd-6df4e1147296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:39.038062 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54ab17-1df8-4246-aebd-6df4e1147296 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:39.038712 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c54ab17-1df8-4246-aebd-6df4e1147296 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:39.040400 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c54ab17-1df8-4246-aebd-6df4e1147296 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:39.053210 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1170/2320] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:44:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:44:39.057397 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a96c0-cabc-4a6c-b29f-e307c8922400 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:39.068148 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a96c0-cabc-4a6c-b29f-e307c8922400 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:39.073043 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b09bf2-eb51-48c5-9860-51a4a1334de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:39.073713 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b09bf2-eb51-48c5-9860-51a4a1334de4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:39.074105 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b09bf2-eb51-48c5-9860-51a4a1334de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:39.074387 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b09bf2-eb51-48c5-9860-51a4a1334de4 admin admin] PATH_INFO: `/v3/projects/51caa602b89b4ef884a887c44a5e530e/users/4f331dff540343eb938bc9c2de9cedcb/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:39.090248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b09bf2-eb51-48c5-9860-51a4a1334de4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51caa602b89b4ef884a887c44a5e530e, user_id=4f331dff540343eb938bc9c2de9cedcb, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:39.092948 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b09bf2-eb51-48c5-9860-51a4a1334de4 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:39.108465 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1151/2321] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:44:39 2022] PUT /identity/v3/projects/51caa602b89b4ef884a887c44a5e530e/users/4f331dff540343eb938bc9c2de9cedcb/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:39.112881 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c54ab17-1df8-4246-aebd-6df4e1147296 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:39.124128 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c54ab17-1df8-4246-aebd-6df4e1147296 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:39.180802 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:39.192162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b8d98f-18e6-4280-b392-6fdf79718a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:39.192772 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b8d98f-18e6-4280-b392-6fdf79718a63 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:39.193049 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b8d98f-18e6-4280-b392-6fdf79718a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:39.193321 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b8d98f-18e6-4280-b392-6fdf79718a63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:39.193954 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b8d98f-18e6-4280-b392-6fdf79718a63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:39.195583 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b8d98f-18e6-4280-b392-6fdf79718a63 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:39.205750 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1171/2322] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:44:39 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:44:39.209928 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b09bf2-eb51-48c5-9860-51a4a1334de4 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:39.221351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b09bf2-eb51-48c5-9860-51a4a1334de4 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:39.225982 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2406714-21c5-4353-9504-c662b640a495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:39.226633 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2406714-21c5-4353-9504-c662b640a495 admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:39.226915 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2406714-21c5-4353-9504-c662b640a495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:39.227183 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2406714-21c5-4353-9504-c662b640a495 admin admin] PATH_INFO: `/v3/projects/51caa602b89b4ef884a887c44a5e530e/users/4f331dff540343eb938bc9c2de9cedcb/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:39.229213 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2406714-21c5-4353-9504-c662b640a495 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51caa602b89b4ef884a887c44a5e530e, user_id=4f331dff540343eb938bc9c2de9cedcb, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:39.232418 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2406714-21c5-4353-9504-c662b640a495 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:39.242764 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1152/2323] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:44:39 2022] PUT /identity/v3/projects/51caa602b89b4ef884a887c44a5e530e/users/4f331dff540343eb938bc9c2de9cedcb/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:40.149401 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b35899e-83b0-4c88-be37-c02456a7703c None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:40.149884 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b35899e-83b0-4c88-be37-c02456a7703c None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:40.149884 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b35899e-83b0-4c88-be37-c02456a7703c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:40.182408 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-6b35899e-83b0-4c88-be37-c02456a7703c None None] MFA Rules not processed for user `4f331dff540343eb938bc9c2de9cedcb`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:40.183768 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-6b35899e-83b0-4c88-be37-c02456a7703c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:40.300091 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1172/2324] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:44:40 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:44:40.315142 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2406714-21c5-4353-9504-c662b640a495 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:40.317373 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-f2406714-21c5-4353-9504-c662b640a495 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:40.390828 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2406714-21c5-4353-9504-c662b640a495 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:40.398727 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e405bdcb-8877-46f4-94f8-95fa4723f1c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:40.399673 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405bdcb-8877-46f4-94f8-95fa4723f1c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:40.400051 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405bdcb-8877-46f4-94f8-95fa4723f1c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:40.400431 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405bdcb-8877-46f4-94f8-95fa4723f1c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:40.461548 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e405bdcb-8877-46f4-94f8-95fa4723f1c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:40.462980 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e405bdcb-8877-46f4-94f8-95fa4723f1c5 service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:40.472006 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1153/2325] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:44:41.491054 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d376ad49-a928-4e97-93d4-0ad2defd190b None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:41.491590 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d376ad49-a928-4e97-93d4-0ad2defd190b None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:41.491778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d376ad49-a928-4e97-93d4-0ad2defd190b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:41.514302 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-d376ad49-a928-4e97-93d4-0ad2defd190b None None] MFA Rules not processed for user `08b4f8865dd14290ab714039a2e424fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:41.515548 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-d376ad49-a928-4e97-93d4-0ad2defd190b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:41.578244 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1173/2326] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:44:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:41.830280 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e405bdcb-8877-46f4-94f8-95fa4723f1c5 service cinder] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:41.831798 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-e405bdcb-8877-46f4-94f8-95fa4723f1c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:41.843523 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e405bdcb-8877-46f4-94f8-95fa4723f1c5 service cinder] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:41.847201 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13827ded-b785-4c79-b9ff-14b786e1b6c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:41.847833 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13827ded-b785-4c79-b9ff-14b786e1b6c4 service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:41.848117 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13827ded-b785-4c79-b9ff-14b786e1b6c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:41.848391 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13827ded-b785-4c79-b9ff-14b786e1b6c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:41.858193 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13827ded-b785-4c79-b9ff-14b786e1b6c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:41.859724 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13827ded-b785-4c79-b9ff-14b786e1b6c4 service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:41.871265 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1154/2327] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:41 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:44:44.630474 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2391bb6e-a57b-403e-ab34-76d5851a6721 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:44.631429 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2391bb6e-a57b-403e-ab34-76d5851a6721 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:44.631986 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2391bb6e-a57b-403e-ab34-76d5851a6721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:44.659404 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-2391bb6e-a57b-403e-ab34-76d5851a6721 None None] MFA Rules not processed for user `f60bee471e954932a928ab2cf7d66c82`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:44.659404 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-2391bb6e-a57b-403e-ab34-76d5851a6721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:44.720840 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1174/2328] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Aug 30 14:44:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:44.788386 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13827ded-b785-4c79-b9ff-14b786e1b6c4 service cinder] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:44.789837 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-13827ded-b785-4c79-b9ff-14b786e1b6c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:44.867606 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:44.887241 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13827ded-b785-4c79-b9ff-14b786e1b6c4 service cinder] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:44.892893 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafe30df-83a0-4e0a-bff8-fc7cdbf02ebf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a18b13e3708144e1b5e8df80ec3a4e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:44.893572 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe30df-83a0-4e0a-bff8-fc7cdbf02ebf service swift] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:44.894159 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe30df-83a0-4e0a-bff8-fc7cdbf02ebf service swift] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:44.894543 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe30df-83a0-4e0a-bff8-fc7cdbf02ebf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:44.903601 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafe30df-83a0-4e0a-bff8-fc7cdbf02ebf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:44.905092 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafe30df-83a0-4e0a-bff8-fc7cdbf02ebf service swift] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:44.915286 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1155/2329] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Aug 30 14:44:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 14:44:45.968915 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2391bb6e-a57b-403e-ab34-76d5851a6721 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:45.970447 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-2391bb6e-a57b-403e-ab34-76d5851a6721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:46.008998 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:46.069361 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2391bb6e-a57b-403e-ab34-76d5851a6721 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:46.078360 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254b95c-86e0-4e78-9fdf-69ec89cc69f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:46.079511 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254b95c-86e0-4e78-9fdf-69ec89cc69f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:46.080023 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254b95c-86e0-4e78-9fdf-69ec89cc69f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:46.080564 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254b95c-86e0-4e78-9fdf-69ec89cc69f9 admin admin] PATH_INFO: `/v3/users/30292db03974498fb3de914e20ba7d09` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:46.082108 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b254b95c-86e0-4e78-9fdf-69ec89cc69f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30292db03974498fb3de914e20ba7d09)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:46.084297 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b254b95c-86e0-4e78-9fdf-69ec89cc69f9 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:46.177431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-b254b95c-86e0-4e78-9fdf-69ec89cc69f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30292db03974498fb3de914e20ba7d09'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:46.178625 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-b254b95c-86e0-4e78-9fdf-69ec89cc69f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30292db03974498fb3de914e20ba7d09'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:46.196683 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-b254b95c-86e0-4e78-9fdf-69ec89cc69f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30292db03974498fb3de914e20ba7d09'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:46.225478 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-b254b95c-86e0-4e78-9fdf-69ec89cc69f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30292db03974498fb3de914e20ba7d09'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:46.226414 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-b254b95c-86e0-4e78-9fdf-69ec89cc69f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30292db03974498fb3de914e20ba7d09'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:46.252219 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1175/2330] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:44:45 2022] DELETE /identity/v3/users/30292db03974498fb3de914e20ba7d09 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:46.257933 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d668d-750a-4cfd-bdb0-171bdc4fdc28 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:46.258508 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d668d-750a-4cfd-bdb0-171bdc4fdc28 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:46.258838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d668d-750a-4cfd-bdb0-171bdc4fdc28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:46.309811 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-179d668d-750a-4cfd-bdb0-171bdc4fdc28 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:46.324974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-179d668d-750a-4cfd-bdb0-171bdc4fdc28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:46.480617 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1156/2331] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:44:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:46.497267 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254b95c-86e0-4e78-9fdf-69ec89cc69f9 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:46.500202 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-b254b95c-86e0-4e78-9fdf-69ec89cc69f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:46.621231 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254b95c-86e0-4e78-9fdf-69ec89cc69f9 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:46.625485 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99de558-7e2d-49b9-9556-c1837388b1e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:46.626109 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99de558-7e2d-49b9-9556-c1837388b1e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:46.626393 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99de558-7e2d-49b9-9556-c1837388b1e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:46.626730 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99de558-7e2d-49b9-9556-c1837388b1e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:46.639421 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99de558-7e2d-49b9-9556-c1837388b1e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:46.641553 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99de558-7e2d-49b9-9556-c1837388b1e2 service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:46.653032 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1176/2332] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:44:47.116789 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179d668d-750a-4cfd-bdb0-171bdc4fdc28 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:47.118471 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-179d668d-750a-4cfd-bdb0-171bdc4fdc28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:47.136099 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179d668d-750a-4cfd-bdb0-171bdc4fdc28 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:47.142661 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff4399f-7608-470f-86b0-c7e3144943da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:47.143399 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff4399f-7608-470f-86b0-c7e3144943da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:47.143682 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff4399f-7608-470f-86b0-c7e3144943da admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:47.143958 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff4399f-7608-470f-86b0-c7e3144943da admin admin] PATH_INFO: `/v3/projects/a49798c20d924459b84ff50b8a44ff08` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:47.145169 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff4399f-7608-470f-86b0-c7e3144943da admin admin] RBAC: Authorizing `identity:delete_project(project_id=a49798c20d924459b84ff50b8a44ff08)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:47.147194 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff4399f-7608-470f-86b0-c7e3144943da admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:47.211200 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-4ff4399f-7608-470f-86b0-c7e3144943da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a49798c20d924459b84ff50b8a44ff08', 'request_id': 'req-4ff4399f-7608-470f-86b0-c7e3144943da'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:47.225229 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-4ff4399f-7608-470f-86b0-c7e3144943da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a49798c20d924459b84ff50b8a44ff08', 'request_id': 'req-4ff4399f-7608-470f-86b0-c7e3144943da'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:47.235574 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-4ff4399f-7608-470f-86b0-c7e3144943da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a49798c20d924459b84ff50b8a44ff08', 'request_id': 'req-4ff4399f-7608-470f-86b0-c7e3144943da'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:47.254776 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-4ff4399f-7608-470f-86b0-c7e3144943da admin admin] The token cache is being invalidate because project a49798c20d924459b84ff50b8a44ff08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:44:47.255455 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-4ff4399f-7608-470f-86b0-c7e3144943da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:47.256280 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-4ff4399f-7608-470f-86b0-c7e3144943da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:47.259305 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1157/2333] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:44:47 2022] DELETE /identity/v3/projects/a49798c20d924459b84ff50b8a44ff08 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:47.264541 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82f378-3593-4ba5-85c1-21318de18f0f None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:47.265536 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82f378-3593-4ba5-85c1-21318de18f0f None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:47.266044 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82f378-3593-4ba5-85c1-21318de18f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:47.305607 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-9d82f378-3593-4ba5-85c1-21318de18f0f None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:47.311921 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-9d82f378-3593-4ba5-85c1-21318de18f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:47.378677 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:47.418974 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1177/2334] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:44:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:47.489678 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e08b96-4401-42ee-9b0b-4dbf3bf6aabe None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:47.489963 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e08b96-4401-42ee-9b0b-4dbf3bf6aabe None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:47.490244 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e08b96-4401-42ee-9b0b-4dbf3bf6aabe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:47.522603 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-19e08b96-4401-42ee-9b0b-4dbf3bf6aabe None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:47.532711 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-19e08b96-4401-42ee-9b0b-4dbf3bf6aabe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:47.547314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1158/2335] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:44:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:47.548564 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d82f378-3593-4ba5-85c1-21318de18f0f None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:47.560138 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d82f378-3593-4ba5-85c1-21318de18f0f None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:47.566297 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20117f68-a878-4786-9377-c719e798eb57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:47.566876 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20117f68-a878-4786-9377-c719e798eb57 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:47.567200 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20117f68-a878-4786-9377-c719e798eb57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:47.567485 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20117f68-a878-4786-9377-c719e798eb57 admin admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:47.568229 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20117f68-a878-4786-9377-c719e798eb57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:44:47.568475 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20117f68-a878-4786-9377-c719e798eb57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:47.570288 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20117f68-a878-4786-9377-c719e798eb57 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:47.585889 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1178/2336] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 14:44:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:44:47.592110 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e08b96-4401-42ee-9b0b-4dbf3bf6aabe None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:47.603379 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e08b96-4401-42ee-9b0b-4dbf3bf6aabe None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:47.612222 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e5e726-6b2b-49bc-b5a7-93bde0276a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:47.612779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5e726-6b2b-49bc-b5a7-93bde0276a06 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:47.613052 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5e726-6b2b-49bc-b5a7-93bde0276a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:47.613497 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5e726-6b2b-49bc-b5a7-93bde0276a06 admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:47.614475 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e5e726-6b2b-49bc-b5a7-93bde0276a06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:47.617067 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e5e726-6b2b-49bc-b5a7-93bde0276a06 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:47.687077 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1159/2337] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:44:47 2022] POST /identity/v3/projects => generated 371 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:44:47.692947 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20117f68-a878-4786-9377-c719e798eb57 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:47.704134 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20117f68-a878-4786-9377-c719e798eb57 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:47.825188 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10179bb3-2a36-4814-8f08-3fd36fa43347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:47.826014 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10179bb3-2a36-4814-8f08-3fd36fa43347 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:47.826637 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10179bb3-2a36-4814-8f08-3fd36fa43347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:47.826987 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10179bb3-2a36-4814-8f08-3fd36fa43347 admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:47.827712 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10179bb3-2a36-4814-8f08-3fd36fa43347 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:47.831965 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10179bb3-2a36-4814-8f08-3fd36fa43347 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:47.877909 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1179/2338] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:44:47 2022] POST /identity/v3/users => generated 286 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:44:47.879303 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e5e726-6b2b-49bc-b5a7-93bde0276a06 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:47.890550 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e5e726-6b2b-49bc-b5a7-93bde0276a06 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:47.900219 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d536f0a-ad98-4638-9e67-b13785891b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:47.900856 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d536f0a-ad98-4638-9e67-b13785891b64 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:47.901134 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d536f0a-ad98-4638-9e67-b13785891b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:47.901410 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d536f0a-ad98-4638-9e67-b13785891b64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:47.902073 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d536f0a-ad98-4638-9e67-b13785891b64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:47.906355 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d536f0a-ad98-4638-9e67-b13785891b64 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:47.922836 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1160/2339] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:44:47 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:44:47.925920 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10179bb3-2a36-4814-8f08-3fd36fa43347 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:47.957938 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10179bb3-2a36-4814-8f08-3fd36fa43347 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:47.982739 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e63ffa2-41fa-4465-a8e0-1a5832b6bdeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:47.983416 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63ffa2-41fa-4465-a8e0-1a5832b6bdeb admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:47.983701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63ffa2-41fa-4465-a8e0-1a5832b6bdeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:47.983965 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63ffa2-41fa-4465-a8e0-1a5832b6bdeb admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:47.984612 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e63ffa2-41fa-4465-a8e0-1a5832b6bdeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:47.986206 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e63ffa2-41fa-4465-a8e0-1a5832b6bdeb admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:48.007626 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1180/2340] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:44:47 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:44:48.008875 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d536f0a-ad98-4638-9e67-b13785891b64 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:48.025970 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d536f0a-ad98-4638-9e67-b13785891b64 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:48.045555 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f194bc4d-2ee8-45c0-9788-1a166a2f6a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:48.046227 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f194bc4d-2ee8-45c0-9788-1a166a2f6a78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:48.046511 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f194bc4d-2ee8-45c0-9788-1a166a2f6a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:48.046786 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f194bc4d-2ee8-45c0-9788-1a166a2f6a78 admin admin] PATH_INFO: `/v3/projects/92ffb73db5f54a6d8d5d1f1e50103d22/users/17ac76bb8f2543409fc2a64541d591a4/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:48.085522 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f194bc4d-2ee8-45c0-9788-1a166a2f6a78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92ffb73db5f54a6d8d5d1f1e50103d22, user_id=17ac76bb8f2543409fc2a64541d591a4, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:48.088926 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f194bc4d-2ee8-45c0-9788-1a166a2f6a78 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:48.128349 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1161/2341] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:44:48 2022] PUT /identity/v3/projects/92ffb73db5f54a6d8d5d1f1e50103d22/users/17ac76bb8f2543409fc2a64541d591a4/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:44:48.130101 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb7e0f-531a-46fc-9d4e-e217549a974b None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:48.130404 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb7e0f-531a-46fc-9d4e-e217549a974b None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:48.130677 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb7e0f-531a-46fc-9d4e-e217549a974b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:48.164085 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-97eb7e0f-531a-46fc-9d4e-e217549a974b None None] MFA Rules not processed for user `17ac76bb8f2543409fc2a64541d591a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:48.165300 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-97eb7e0f-531a-46fc-9d4e-e217549a974b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:48.286021 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1181/2342] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:44:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:48.294461 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f194bc4d-2ee8-45c0-9788-1a166a2f6a78 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:48.296223 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-f194bc4d-2ee8-45c0-9788-1a166a2f6a78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:48.305985 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:48.373756 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f194bc4d-2ee8-45c0-9788-1a166a2f6a78 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:48.379077 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee6b5d9-9711-4226-b5f4-75e4c6079d8c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6889402e54424bbd8d42c7bb759cf700', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:48.379757 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6b5d9-9711-4226-b5f4-75e4c6079d8c service glance] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:48.380072 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6b5d9-9711-4226-b5f4-75e4c6079d8c service glance] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:48.380296 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6b5d9-9711-4226-b5f4-75e4c6079d8c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:48.390451 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee6b5d9-9711-4226-b5f4-75e4c6079d8c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:48.392015 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee6b5d9-9711-4226-b5f4-75e4c6079d8c service glance] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:48.410131 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1162/2343] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:48 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:44:56.350945 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd92da-1534-45a8-b0ec-eb6f562dcb46 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:56.351599 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd92da-1534-45a8-b0ec-eb6f562dcb46 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:56.351599 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd92da-1534-45a8-b0ec-eb6f562dcb46 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:56.352881 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1182/2344] 10.222.0.30 () {54 vars in 921 bytes} [Tue Aug 30 14:44:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 14:44:56.385680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7d842-12a3-4fd5-871c-f43a56edebda None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:56.386808 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7d842-12a3-4fd5-871c-f43a56edebda None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:56.387272 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7d842-12a3-4fd5-871c-f43a56edebda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:56.458917 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-9cf7d842-12a3-4fd5-871c-f43a56edebda None None] MFA Rules not processed for user `dcf302a170934c25b39699088dac326a`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:56.462687 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-9cf7d842-12a3-4fd5-871c-f43a56edebda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:56.561141 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1163/2345] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Aug 30 14:44:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:56.567083 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd92da-1534-45a8-b0ec-eb6f562dcb46 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:56.569310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-b6fd92da-1534-45a8-b0ec-eb6f562dcb46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:56.590058 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd92da-1534-45a8-b0ec-eb6f562dcb46 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:56.597541 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d1465f-226d-40cc-833a-30205b986e73 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcf302a170934c25b39699088dac326a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:56.598450 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d1465f-226d-40cc-833a-30205b986e73 service placement] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:56.598969 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d1465f-226d-40cc-833a-30205b986e73 service placement] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:56.599500 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d1465f-226d-40cc-833a-30205b986e73 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:56.612981 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d1465f-226d-40cc-833a-30205b986e73 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:56.617482 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d1465f-226d-40cc-833a-30205b986e73 service placement] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:56.653513 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1183/2346] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:44:57.263758 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf7d842-12a3-4fd5-871c-f43a56edebda None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:57.265456 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-9cf7d842-12a3-4fd5-871c-f43a56edebda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:57.343707 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf7d842-12a3-4fd5-871c-f43a56edebda None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:57.349405 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab1dc7-2248-4e41-aebb-788ed8e912db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:57.350252 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab1dc7-2248-4e41-aebb-788ed8e912db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:57.350553 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab1dc7-2248-4e41-aebb-788ed8e912db admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:57.350826 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab1dc7-2248-4e41-aebb-788ed8e912db admin admin] PATH_INFO: `/v3/users/65539b4338a74f53af7d4813f6d6ac44` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:57.351990 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ab1dc7-2248-4e41-aebb-788ed8e912db admin admin] RBAC: Authorizing `identity:delete_user(user_id=65539b4338a74f53af7d4813f6d6ac44)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:57.354689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ab1dc7-2248-4e41-aebb-788ed8e912db admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:57.406985 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:57.409148 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-08ab1dc7-2248-4e41-aebb-788ed8e912db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65539b4338a74f53af7d4813f6d6ac44'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:57.410042 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-08ab1dc7-2248-4e41-aebb-788ed8e912db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65539b4338a74f53af7d4813f6d6ac44'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:57.425389 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-08ab1dc7-2248-4e41-aebb-788ed8e912db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65539b4338a74f53af7d4813f6d6ac44'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:57.433594 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-08ab1dc7-2248-4e41-aebb-788ed8e912db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65539b4338a74f53af7d4813f6d6ac44'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:57.443740 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-08ab1dc7-2248-4e41-aebb-788ed8e912db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65539b4338a74f53af7d4813f6d6ac44'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:57.450927 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1164/2347] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:44:57 2022] DELETE /identity/v3/users/65539b4338a74f53af7d4813f6d6ac44 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:57.841055 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d1465f-226d-40cc-833a-30205b986e73 service placement] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:57.851770 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d1465f-226d-40cc-833a-30205b986e73 service placement] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:57.877737 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:57.935455 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37a0a6a-a249-49ea-b80a-c0434163e662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:57.936388 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a0a6a-a249-49ea-b80a-c0434163e662 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:57.936674 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a0a6a-a249-49ea-b80a-c0434163e662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:57.937185 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a0a6a-a249-49ea-b80a-c0434163e662 admin admin] PATH_INFO: `/v3/projects/fd26aaeccd554bb9923f4287cf9e06da` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:57.940585 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37a0a6a-a249-49ea-b80a-c0434163e662 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd26aaeccd554bb9923f4287cf9e06da)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:57.945254 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37a0a6a-a249-49ea-b80a-c0434163e662 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:58.004590 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-b37a0a6a-a249-49ea-b80a-c0434163e662 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd26aaeccd554bb9923f4287cf9e06da', 'request_id': 'req-b37a0a6a-a249-49ea-b80a-c0434163e662'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:58.011263 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-b37a0a6a-a249-49ea-b80a-c0434163e662 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd26aaeccd554bb9923f4287cf9e06da', 'request_id': 'req-b37a0a6a-a249-49ea-b80a-c0434163e662'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:58.017990 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-b37a0a6a-a249-49ea-b80a-c0434163e662 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd26aaeccd554bb9923f4287cf9e06da', 'request_id': 'req-b37a0a6a-a249-49ea-b80a-c0434163e662'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:58.044245 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-b37a0a6a-a249-49ea-b80a-c0434163e662 admin admin] The token cache is being invalidate because project fd26aaeccd554bb9923f4287cf9e06da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:44:58.044551 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-b37a0a6a-a249-49ea-b80a-c0434163e662 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:58.045237 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-b37a0a6a-a249-49ea-b80a-c0434163e662 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:58.047097 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1184/2348] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:44:57 2022] DELETE /identity/v3/projects/fd26aaeccd554bb9923f4287cf9e06da => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:58.051142 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab1dc7-2248-4e41-aebb-788ed8e912db admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:58.052625 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-08ab1dc7-2248-4e41-aebb-788ed8e912db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:58.067862 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab1dc7-2248-4e41-aebb-788ed8e912db admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:58.173285 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f19ac9-315f-4a8c-af77-874c752d1398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:58.173930 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f19ac9-315f-4a8c-af77-874c752d1398 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:58.174211 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f19ac9-315f-4a8c-af77-874c752d1398 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:58.174487 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f19ac9-315f-4a8c-af77-874c752d1398 admin admin] PATH_INFO: `/v3/users/4f331dff540343eb938bc9c2de9cedcb` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:58.175656 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f19ac9-315f-4a8c-af77-874c752d1398 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f331dff540343eb938bc9c2de9cedcb)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:58.177600 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f19ac9-315f-4a8c-af77-874c752d1398 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:58.251746 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-81f19ac9-315f-4a8c-af77-874c752d1398 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f331dff540343eb938bc9c2de9cedcb'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:58.252410 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-81f19ac9-315f-4a8c-af77-874c752d1398 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f331dff540343eb938bc9c2de9cedcb'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:58.265231 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-81f19ac9-315f-4a8c-af77-874c752d1398 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f331dff540343eb938bc9c2de9cedcb'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:58.272464 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-81f19ac9-315f-4a8c-af77-874c752d1398 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f331dff540343eb938bc9c2de9cedcb'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:58.280177 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-81f19ac9-315f-4a8c-af77-874c752d1398 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f331dff540343eb938bc9c2de9cedcb'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:58.286536 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1165/2349] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:44:58 2022] DELETE /identity/v3/users/4f331dff540343eb938bc9c2de9cedcb => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:44:58.476593 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37a0a6a-a249-49ea-b80a-c0434163e662 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:58.479546 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-b37a0a6a-a249-49ea-b80a-c0434163e662 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:58.498738 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37a0a6a-a249-49ea-b80a-c0434163e662 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:58.548141 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b984fd2-667c-4f33-91e7-09338eee6d09 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:58.548650 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b984fd2-667c-4f33-91e7-09338eee6d09 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:58.548747 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b984fd2-667c-4f33-91e7-09338eee6d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:58.570495 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-7b984fd2-667c-4f33-91e7-09338eee6d09 None None] MFA Rules not processed for user `35f28006916040c3b988de6c15093166`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:58.571745 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-7b984fd2-667c-4f33-91e7-09338eee6d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:58.574125 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a1dbdb-78b1-4e04-b8b6-38313586850b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:58.575064 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a1dbdb-78b1-4e04-b8b6-38313586850b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:58.575373 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a1dbdb-78b1-4e04-b8b6-38313586850b admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:58.575650 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a1dbdb-78b1-4e04-b8b6-38313586850b admin admin] PATH_INFO: `/v3/projects/51caa602b89b4ef884a887c44a5e530e` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:58.576808 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a1dbdb-78b1-4e04-b8b6-38313586850b admin admin] RBAC: Authorizing `identity:delete_project(project_id=51caa602b89b4ef884a887c44a5e530e)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:58.578749 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a1dbdb-78b1-4e04-b8b6-38313586850b admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:58.611505 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:58.628065 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-10a1dbdb-78b1-4e04-b8b6-38313586850b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51caa602b89b4ef884a887c44a5e530e', 'request_id': 'req-10a1dbdb-78b1-4e04-b8b6-38313586850b'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:58.631960 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1166/2350] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:44:58 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:44:58.634039 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-10a1dbdb-78b1-4e04-b8b6-38313586850b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51caa602b89b4ef884a887c44a5e530e', 'request_id': 'req-10a1dbdb-78b1-4e04-b8b6-38313586850b'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:58.635340 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:58.643133 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-10a1dbdb-78b1-4e04-b8b6-38313586850b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51caa602b89b4ef884a887c44a5e530e', 'request_id': 'req-10a1dbdb-78b1-4e04-b8b6-38313586850b'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:58.677059 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-10a1dbdb-78b1-4e04-b8b6-38313586850b admin admin] The token cache is being invalidate because project 51caa602b89b4ef884a887c44a5e530e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:44:58.677353 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-10a1dbdb-78b1-4e04-b8b6-38313586850b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:58.678994 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-10a1dbdb-78b1-4e04-b8b6-38313586850b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:58.680952 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1185/2351] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:44:58 2022] DELETE /identity/v3/projects/51caa602b89b4ef884a887c44a5e530e => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:58.687473 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d4776-8849-4978-a36a-1af76a5ad21d None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:58.687763 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d4776-8849-4978-a36a-1af76a5ad21d None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:58.688138 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d4776-8849-4978-a36a-1af76a5ad21d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:58.709843 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-748d4776-8849-4978-a36a-1af76a5ad21d None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:58.711037 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-748d4776-8849-4978-a36a-1af76a5ad21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:58.793537 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1167/2352] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:44:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:58.796552 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d33114d-defd-4643-9b66-94eac46b677e None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:58.796831 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d33114d-defd-4643-9b66-94eac46b677e None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:58.797099 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d33114d-defd-4643-9b66-94eac46b677e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:58.821908 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-2d33114d-defd-4643-9b66-94eac46b677e None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:58.823322 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-2d33114d-defd-4643-9b66-94eac46b677e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:58.832304 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1186/2353] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:44:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:58.836778 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d4776-8849-4978-a36a-1af76a5ad21d None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:58.847901 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d4776-8849-4978-a36a-1af76a5ad21d None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:58.853174 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c1c7e-eb72-4df9-ad8b-eaeb34808501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:58.854017 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c1c7e-eb72-4df9-ad8b-eaeb34808501 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:58.854398 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c1c7e-eb72-4df9-ad8b-eaeb34808501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:58.854785 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c1c7e-eb72-4df9-ad8b-eaeb34808501 admin admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:58.855722 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1c1c7e-eb72-4df9-ad8b-eaeb34808501 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:44:58.856122 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1c1c7e-eb72-4df9-ad8b-eaeb34808501 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:58.857746 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1c1c7e-eb72-4df9-ad8b-eaeb34808501 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:58.875333 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1168/2354] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 14:44:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:44:58.877568 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d33114d-defd-4643-9b66-94eac46b677e None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:58.889737 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d33114d-defd-4643-9b66-94eac46b677e None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:58.895341 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d9c406-66fe-490f-8942-f6ef75871e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:58.896158 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9c406-66fe-490f-8942-f6ef75871e5d admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:58.896652 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9c406-66fe-490f-8942-f6ef75871e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:58.896999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9c406-66fe-490f-8942-f6ef75871e5d admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:58.897724 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d9c406-66fe-490f-8942-f6ef75871e5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:58.899724 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d9c406-66fe-490f-8942-f6ef75871e5d admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:58.932325 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1187/2355] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:44:58 2022] POST /identity/v3/projects => generated 361 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:44:58.933748 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c1c7e-eb72-4df9-ad8b-eaeb34808501 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:58.944281 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c1c7e-eb72-4df9-ad8b-eaeb34808501 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:59.009129 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aab9754-29da-45aa-9632-b63af7fe08d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:59.009742 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab9754-29da-45aa-9632-b63af7fe08d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:59.010597 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab9754-29da-45aa-9632-b63af7fe08d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:59.010877 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab9754-29da-45aa-9632-b63af7fe08d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:59.011564 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aab9754-29da-45aa-9632-b63af7fe08d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:59.013489 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aab9754-29da-45aa-9632-b63af7fe08d7 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:59.038951 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1169/2356] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:44:58 2022] POST /identity/v3/users => generated 281 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:44:59.040381 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d9c406-66fe-490f-8942-f6ef75871e5d admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:59.051729 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d9c406-66fe-490f-8942-f6ef75871e5d admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:59.056585 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c07c8-6188-4131-90b2-f5f90e36b510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:59.057812 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c07c8-6188-4131-90b2-f5f90e36b510 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:59.058092 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c07c8-6188-4131-90b2-f5f90e36b510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:59.058362 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c07c8-6188-4131-90b2-f5f90e36b510 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:59.059012 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700c07c8-6188-4131-90b2-f5f90e36b510 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:59.060679 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700c07c8-6188-4131-90b2-f5f90e36b510 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:59.072897 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1188/2357] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:44:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:44:59.077154 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aab9754-29da-45aa-9632-b63af7fe08d7 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:59.087960 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aab9754-29da-45aa-9632-b63af7fe08d7 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:59.093161 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32994062-730b-4fbf-8e69-8a990343c3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:59.093927 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32994062-730b-4fbf-8e69-8a990343c3d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:59.094207 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32994062-730b-4fbf-8e69-8a990343c3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:59.094477 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32994062-730b-4fbf-8e69-8a990343c3d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:59.095128 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32994062-730b-4fbf-8e69-8a990343c3d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:59.096811 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32994062-730b-4fbf-8e69-8a990343c3d8 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:59.111448 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1170/2358] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:44:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:44:59.115730 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c07c8-6188-4131-90b2-f5f90e36b510 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:59.129367 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c07c8-6188-4131-90b2-f5f90e36b510 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:59.134174 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9690906-ce73-4ba9-b8e8-34867027fca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:59.135045 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9690906-ce73-4ba9-b8e8-34867027fca4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:59.135423 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9690906-ce73-4ba9-b8e8-34867027fca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:59.135871 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9690906-ce73-4ba9-b8e8-34867027fca4 admin admin] PATH_INFO: `/v3/projects/55f49559923946a0a56241d9f2ab3416/users/9ef3106ae30d40dead9054bd3afc390a/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:59.152615 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9690906-ce73-4ba9-b8e8-34867027fca4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55f49559923946a0a56241d9f2ab3416, user_id=9ef3106ae30d40dead9054bd3afc390a, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:59.155338 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9690906-ce73-4ba9-b8e8-34867027fca4 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:59.170771 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1189/2359] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:44:59 2022] PUT /identity/v3/projects/55f49559923946a0a56241d9f2ab3416/users/9ef3106ae30d40dead9054bd3afc390a/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:59.181258 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f94d0b-3030-4445-b9f8-e459a7048eac None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:59.181570 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f94d0b-3030-4445-b9f8-e459a7048eac None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:59.182431 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f94d0b-3030-4445-b9f8-e459a7048eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:59.212203 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-20f94d0b-3030-4445-b9f8-e459a7048eac None None] MFA Rules not processed for user `9ef3106ae30d40dead9054bd3afc390a`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:59.213517 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-20f94d0b-3030-4445-b9f8-e459a7048eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:59.270883 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:44:59.302032 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1171/2360] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:44:59 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:44:59.341971 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9690906-ce73-4ba9-b8e8-34867027fca4 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:59.343814 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-d9690906-ce73-4ba9-b8e8-34867027fca4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:59.439147 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9690906-ce73-4ba9-b8e8-34867027fca4 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:59.449180 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a482bdc-4810-4d4a-879a-9e17bd7e6257 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:59.449820 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a482bdc-4810-4d4a-879a-9e17bd7e6257 service cinder] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:59.450096 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a482bdc-4810-4d4a-879a-9e17bd7e6257 service cinder] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:59.450373 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a482bdc-4810-4d4a-879a-9e17bd7e6257 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:59.467661 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a482bdc-4810-4d4a-879a-9e17bd7e6257 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:59.469375 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a482bdc-4810-4d4a-879a-9e17bd7e6257 service cinder] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:59.484526 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1190/2361] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:44:59.501369 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f94d0b-3030-4445-b9f8-e459a7048eac None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:59.613045 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f94d0b-3030-4445-b9f8-e459a7048eac None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:59.619362 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48b7ad6-13ac-40e1-9bff-1e2f6bc232f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:59.620375 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b7ad6-13ac-40e1-9bff-1e2f6bc232f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:59.620628 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b7ad6-13ac-40e1-9bff-1e2f6bc232f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:59.620905 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b7ad6-13ac-40e1-9bff-1e2f6bc232f0 admin admin] PATH_INFO: `/v3/users/9ef3106ae30d40dead9054bd3afc390a` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:59.622178 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48b7ad6-13ac-40e1-9bff-1e2f6bc232f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ef3106ae30d40dead9054bd3afc390a)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:59.624226 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48b7ad6-13ac-40e1-9bff-1e2f6bc232f0 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:59.685272 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-a48b7ad6-13ac-40e1-9bff-1e2f6bc232f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ef3106ae30d40dead9054bd3afc390a'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:59.686151 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-a48b7ad6-13ac-40e1-9bff-1e2f6bc232f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ef3106ae30d40dead9054bd3afc390a'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:59.698912 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-a48b7ad6-13ac-40e1-9bff-1e2f6bc232f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ef3106ae30d40dead9054bd3afc390a'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:59.707256 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-a48b7ad6-13ac-40e1-9bff-1e2f6bc232f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ef3106ae30d40dead9054bd3afc390a'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:59.725498 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-a48b7ad6-13ac-40e1-9bff-1e2f6bc232f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ef3106ae30d40dead9054bd3afc390a'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:44:59.735099 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1172/2362] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:44:59 2022] DELETE /identity/v3/users/9ef3106ae30d40dead9054bd3afc390a => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:44:59.739644 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ed547-b908-48f3-a483-a770660688e3 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:59.739644 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ed547-b908-48f3-a483-a770660688e3 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:59.739861 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ed547-b908-48f3-a483-a770660688e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:59.763342 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-c24ed547-b908-48f3-a483-a770660688e3 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:44:59.765296 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-c24ed547-b908-48f3-a483-a770660688e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:59.832455 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1191/2363] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:44:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:44:59.843418 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48b7ad6-13ac-40e1-9bff-1e2f6bc232f0 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:44:59.845258 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-a48b7ad6-13ac-40e1-9bff-1e2f6bc232f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:44:59.919133 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48b7ad6-13ac-40e1-9bff-1e2f6bc232f0 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:44:59.924370 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38680323-864b-40fa-b327-2b8dd9a61bcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:44:59.925069 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38680323-864b-40fa-b327-2b8dd9a61bcb service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:44:59.925422 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38680323-864b-40fa-b327-2b8dd9a61bcb service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:44:59.925903 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38680323-864b-40fa-b327-2b8dd9a61bcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:44:59.937505 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38680323-864b-40fa-b327-2b8dd9a61bcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:44:59.939248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38680323-864b-40fa-b327-2b8dd9a61bcb service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:44:59.950964 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1173/2364] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:44:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:45:00.368622 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24ed547-b908-48f3-a483-a770660688e3 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:00.370962 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-c24ed547-b908-48f3-a483-a770660688e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:00.384743 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24ed547-b908-48f3-a483-a770660688e3 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:00.390242 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad10dc1e-fab4-495a-8036-86cbe2084c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:00.390903 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10dc1e-fab4-495a-8036-86cbe2084c1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:00.391178 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10dc1e-fab4-495a-8036-86cbe2084c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:00.391515 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10dc1e-fab4-495a-8036-86cbe2084c1c admin admin] PATH_INFO: `/v3/projects/55f49559923946a0a56241d9f2ab3416` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:00.392656 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad10dc1e-fab4-495a-8036-86cbe2084c1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=55f49559923946a0a56241d9f2ab3416)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:00.394888 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad10dc1e-fab4-495a-8036-86cbe2084c1c admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:00.441521 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:00.480380 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-ad10dc1e-fab4-495a-8036-86cbe2084c1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55f49559923946a0a56241d9f2ab3416', 'request_id': 'req-ad10dc1e-fab4-495a-8036-86cbe2084c1c'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:00.486240 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-ad10dc1e-fab4-495a-8036-86cbe2084c1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55f49559923946a0a56241d9f2ab3416', 'request_id': 'req-ad10dc1e-fab4-495a-8036-86cbe2084c1c'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:00.494694 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-ad10dc1e-fab4-495a-8036-86cbe2084c1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55f49559923946a0a56241d9f2ab3416', 'request_id': 'req-ad10dc1e-fab4-495a-8036-86cbe2084c1c'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:00.508354 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-ad10dc1e-fab4-495a-8036-86cbe2084c1c admin admin] The token cache is being invalidate because project 55f49559923946a0a56241d9f2ab3416 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:45:00.508645 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-ad10dc1e-fab4-495a-8036-86cbe2084c1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:00.509312 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-ad10dc1e-fab4-495a-8036-86cbe2084c1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:00.510982 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1192/2365] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:45:00 2022] DELETE /identity/v3/projects/55f49559923946a0a56241d9f2ab3416 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:00.516042 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c0e5b-0409-4647-9903-639233a00563 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:00.516328 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c0e5b-0409-4647-9903-639233a00563 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:00.516601 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c0e5b-0409-4647-9903-639233a00563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:00.526963 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:00.539285 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-826c0e5b-0409-4647-9903-639233a00563 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:00.542101 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-826c0e5b-0409-4647-9903-639233a00563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:00.622633 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1174/2366] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:45:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:00.630620 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22402759-79a9-4299-9460-377f5c60399d None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:00.630972 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22402759-79a9-4299-9460-377f5c60399d None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:00.631349 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22402759-79a9-4299-9460-377f5c60399d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:00.654654 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-22402759-79a9-4299-9460-377f5c60399d None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:00.655931 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-22402759-79a9-4299-9460-377f5c60399d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:00.664655 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1193/2367] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:45:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:00.669125 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826c0e5b-0409-4647-9903-639233a00563 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:00.679690 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826c0e5b-0409-4647-9903-639233a00563 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:00.685686 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b532d6-8d8a-47f1-baa9-55809f8a886d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:00.687147 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b532d6-8d8a-47f1-baa9-55809f8a886d admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:00.687147 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b532d6-8d8a-47f1-baa9-55809f8a886d admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:00.687147 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b532d6-8d8a-47f1-baa9-55809f8a886d admin admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:00.687992 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b532d6-8d8a-47f1-baa9-55809f8a886d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:45:00.688376 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b532d6-8d8a-47f1-baa9-55809f8a886d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:00.690060 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b532d6-8d8a-47f1-baa9-55809f8a886d admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:00.716770 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1175/2368] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 14:45:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:45:00.717875 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22402759-79a9-4299-9460-377f5c60399d None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:00.727486 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22402759-79a9-4299-9460-377f5c60399d None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:00.732373 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfdbbd3-b5b1-44c6-bf0d-87f78d789d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:00.732892 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfdbbd3-b5b1-44c6-bf0d-87f78d789d9b admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:00.733168 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfdbbd3-b5b1-44c6-bf0d-87f78d789d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:00.733435 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfdbbd3-b5b1-44c6-bf0d-87f78d789d9b admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:00.734108 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfdbbd3-b5b1-44c6-bf0d-87f78d789d9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:00.736213 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfdbbd3-b5b1-44c6-bf0d-87f78d789d9b admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:00.771521 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1194/2369] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:45:00 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 14:45:00.772671 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b532d6-8d8a-47f1-baa9-55809f8a886d admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:00.783846 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b532d6-8d8a-47f1-baa9-55809f8a886d admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:00.856297 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a01b7-1e71-46a8-9623-ec5e61fbe75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:00.857111 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a01b7-1e71-46a8-9623-ec5e61fbe75e admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:00.857476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a01b7-1e71-46a8-9623-ec5e61fbe75e admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:00.857815 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a01b7-1e71-46a8-9623-ec5e61fbe75e admin admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:00.858548 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883a01b7-1e71-46a8-9623-ec5e61fbe75e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:00.860837 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883a01b7-1e71-46a8-9623-ec5e61fbe75e admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:00.888206 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1176/2370] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:45:00 2022] POST /identity/v3/users => generated 287 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:45:00.889426 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfdbbd3-b5b1-44c6-bf0d-87f78d789d9b admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:00.900293 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfdbbd3-b5b1-44c6-bf0d-87f78d789d9b admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:00.910361 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd75e1b-1078-4376-9680-b7ef34c6bc16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:00.911124 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd75e1b-1078-4376-9680-b7ef34c6bc16 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:00.911798 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd75e1b-1078-4376-9680-b7ef34c6bc16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:00.912153 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd75e1b-1078-4376-9680-b7ef34c6bc16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:00.912918 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd75e1b-1078-4376-9680-b7ef34c6bc16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:00.914951 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd75e1b-1078-4376-9680-b7ef34c6bc16 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:00.926668 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1195/2371] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:45:00 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:45:00.930784 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a01b7-1e71-46a8-9623-ec5e61fbe75e admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:00.942034 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a01b7-1e71-46a8-9623-ec5e61fbe75e admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:00.947019 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a40349-046d-400f-b328-4feb9c062e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:00.948495 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a40349-046d-400f-b328-4feb9c062e34 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:00.948776 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a40349-046d-400f-b328-4feb9c062e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:00.949050 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a40349-046d-400f-b328-4feb9c062e34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:00.949704 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a40349-046d-400f-b328-4feb9c062e34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:00.951349 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a40349-046d-400f-b328-4feb9c062e34 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:00.966436 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1177/2372] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:45:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:45:00.967933 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd75e1b-1078-4376-9680-b7ef34c6bc16 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:00.979058 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd75e1b-1078-4376-9680-b7ef34c6bc16 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:00.984323 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377fb678-3b6e-44a7-af59-160c3c0da8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:00.984935 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377fb678-3b6e-44a7-af59-160c3c0da8d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:00.985212 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377fb678-3b6e-44a7-af59-160c3c0da8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:00.985483 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377fb678-3b6e-44a7-af59-160c3c0da8d4 admin admin] PATH_INFO: `/v3/projects/e67c28ec4d3949dc9d00e70f61b5657f/users/a5dd8cf0a16345c5b2973552f0881893/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:01.001917 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377fb678-3b6e-44a7-af59-160c3c0da8d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e67c28ec4d3949dc9d00e70f61b5657f, user_id=a5dd8cf0a16345c5b2973552f0881893, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:01.004741 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377fb678-3b6e-44a7-af59-160c3c0da8d4 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:01.021586 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1196/2373] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:45:00 2022] PUT /identity/v3/projects/e67c28ec4d3949dc9d00e70f61b5657f/users/a5dd8cf0a16345c5b2973552f0881893/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:01.030054 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb40f2b-9113-4228-a91c-b4cf6f249655 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:01.030524 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb40f2b-9113-4228-a91c-b4cf6f249655 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:01.031244 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb40f2b-9113-4228-a91c-b4cf6f249655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:01.067277 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-3eb40f2b-9113-4228-a91c-b4cf6f249655 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:01.068511 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-3eb40f2b-9113-4228-a91c-b4cf6f249655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:01.137909 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1178/2374] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:45:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:01.164236 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377fb678-3b6e-44a7-af59-160c3c0da8d4 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:01.170106 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-377fb678-3b6e-44a7-af59-160c3c0da8d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:01.292439 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377fb678-3b6e-44a7-af59-160c3c0da8d4 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:01.300097 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b08afa-9f99-4a96-a806-43ad67ae07dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:01.300726 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b08afa-9f99-4a96-a806-43ad67ae07dd service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:01.301001 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b08afa-9f99-4a96-a806-43ad67ae07dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:01.301268 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b08afa-9f99-4a96-a806-43ad67ae07dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:01.321562 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b08afa-9f99-4a96-a806-43ad67ae07dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:01.323121 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b08afa-9f99-4a96-a806-43ad67ae07dd service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:01.336153 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1197/2375] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:45:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:45:02.094242 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f691b77b-e16b-4154-9a04-b3dfece3b448 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:02.094791 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f691b77b-e16b-4154-9a04-b3dfece3b448 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:02.094876 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f691b77b-e16b-4154-9a04-b3dfece3b448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:02.116273 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:02.126618 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-f691b77b-e16b-4154-9a04-b3dfece3b448 None None] MFA Rules not processed for user `a5dd8cf0a16345c5b2973552f0881893`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:02.128760 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-f691b77b-e16b-4154-9a04-b3dfece3b448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:02.208768 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1179/2376] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:45:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:02.216979 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b08afa-9f99-4a96-a806-43ad67ae07dd service neutron] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:02.219902 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-e7b08afa-9f99-4a96-a806-43ad67ae07dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:02.284035 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:02.295656 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b08afa-9f99-4a96-a806-43ad67ae07dd service neutron] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:02.300651 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bea2c35-70d7-492e-bb59-49d677257c2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:02.311430 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea2c35-70d7-492e-bb59-49d677257c2d service cinder] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:02.311720 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea2c35-70d7-492e-bb59-49d677257c2d service cinder] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:02.311993 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea2c35-70d7-492e-bb59-49d677257c2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:02.339913 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bea2c35-70d7-492e-bb59-49d677257c2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:02.341726 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bea2c35-70d7-492e-bb59-49d677257c2d service cinder] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:02.351609 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1198/2377] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:45:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:45:03.755657 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b136d-dce3-4163-b77d-b0388e8db25c None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:03.756219 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b136d-dce3-4163-b77d-b0388e8db25c None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:03.756219 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b136d-dce3-4163-b77d-b0388e8db25c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:03.778291 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-904b136d-dce3-4163-b77d-b0388e8db25c None None] MFA Rules not processed for user `a5dd8cf0a16345c5b2973552f0881893`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:03.779697 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-904b136d-dce3-4163-b77d-b0388e8db25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:03.792859 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1180/2378] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:45:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:05.783431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b35055-9cb4-473f-b272-f140faee2dd0 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:05.783924 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b35055-9cb4-473f-b272-f140faee2dd0 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:05.784043 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b35055-9cb4-473f-b272-f140faee2dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:05.808136 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-f5b35055-9cb4-473f-b272-f140faee2dd0 None None] MFA Rules not processed for user `f60bee471e954932a928ab2cf7d66c82`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:05.809802 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-f5b35055-9cb4-473f-b272-f140faee2dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:05.882148 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1199/2379] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Aug 30 14:45:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:05.906793 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904b136d-dce3-4163-b77d-b0388e8db25c None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:05.908283 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-904b136d-dce3-4163-b77d-b0388e8db25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:05.915455 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93466485-e12f-41fe-b2ad-e9cd77a27002 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:05.916381 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93466485-e12f-41fe-b2ad-e9cd77a27002 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:05.917162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93466485-e12f-41fe-b2ad-e9cd77a27002 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:05.918610 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1200/2380] 10.222.0.30 () {54 vars in 921 bytes} [Tue Aug 30 14:45:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:45:05.924547 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b78a99-860f-44c2-8116-3928661e0f06 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:05.924831 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b78a99-860f-44c2-8116-3928661e0f06 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:05.925106 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b78a99-860f-44c2-8116-3928661e0f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:05.945916 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-c7b78a99-860f-44c2-8116-3928661e0f06 None None] MFA Rules not processed for user `8878de66a03044e38e507028853a0cd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:05.947193 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-c7b78a99-860f-44c2-8116-3928661e0f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:05.992297 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904b136d-dce3-4163-b77d-b0388e8db25c None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:06.000600 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2246e565-6075-4bf8-afaa-3da63187565f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a18b13e3708144e1b5e8df80ec3a4e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:06.001251 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2246e565-6075-4bf8-afaa-3da63187565f service swift] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:06.001530 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2246e565-6075-4bf8-afaa-3da63187565f service swift] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:06.001964 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2246e565-6075-4bf8-afaa-3da63187565f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:06.015665 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2246e565-6075-4bf8-afaa-3da63187565f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:06.017602 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2246e565-6075-4bf8-afaa-3da63187565f service swift] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:06.028876 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1201/2381] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Aug 30 14:45:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:06.039101 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1181/2382] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Aug 30 14:45:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 14:45:06.047707 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b78a99-860f-44c2-8116-3928661e0f06 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:06.049675 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-c7b78a99-860f-44c2-8116-3928661e0f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:06.115911 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b78a99-860f-44c2-8116-3928661e0f06 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:06.119945 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eff6e0-516b-4410-8a90-4a42a8424be4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6889402e54424bbd8d42c7bb759cf700', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:06.120620 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eff6e0-516b-4410-8a90-4a42a8424be4 service glance] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:06.120901 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eff6e0-516b-4410-8a90-4a42a8424be4 service glance] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:06.121175 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eff6e0-516b-4410-8a90-4a42a8424be4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:06.133354 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eff6e0-516b-4410-8a90-4a42a8424be4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:06.135652 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eff6e0-516b-4410-8a90-4a42a8424be4 service glance] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:06.156744 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1202/2383] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:45:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 14:45:06.242127 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b30dff-28e1-4b92-bd47-aa8837071952 None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:06.242420 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b30dff-28e1-4b92-bd47-aa8837071952 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:06.242697 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b30dff-28e1-4b92-bd47-aa8837071952 None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:06.244676 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1182/2384] 10.222.0.30 () {54 vars in 921 bytes} [Tue Aug 30 14:45:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:45:06.250122 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8c82c-98fc-4b56-9309-8767a9b0e385 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:06.250420 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8c82c-98fc-4b56-9309-8767a9b0e385 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:06.250789 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8c82c-98fc-4b56-9309-8767a9b0e385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:06.272746 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-07c8c82c-98fc-4b56-9309-8767a9b0e385 None None] MFA Rules not processed for user `9e14e86e085d4987aece69fe64867ce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:06.273924 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-07c8c82c-98fc-4b56-9309-8767a9b0e385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:06.284879 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1203/2385] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Aug 30 14:45:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:06.302454 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b30dff-28e1-4b92-bd47-aa8837071952 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:06.314096 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b30dff-28e1-4b92-bd47-aa8837071952 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:06.319353 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8231d9f-13d3-4d9c-8c9e-757a46353665 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:06.319353 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8231d9f-13d3-4d9c-8c9e-757a46353665 service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:06.319353 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8231d9f-13d3-4d9c-8c9e-757a46353665 service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:06.319622 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8231d9f-13d3-4d9c-8c9e-757a46353665 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:06.327796 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8231d9f-13d3-4d9c-8c9e-757a46353665 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:06.329906 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8231d9f-13d3-4d9c-8c9e-757a46353665 service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:06.339452 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1183/2386] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:45:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:45:06.501405 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c95387-f481-470e-81b7-3579d73fe083 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:06.501760 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c95387-f481-470e-81b7-3579d73fe083 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:06.502038 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c95387-f481-470e-81b7-3579d73fe083 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:06.504465 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1204/2387] 10.222.0.30 () {54 vars in 921 bytes} [Tue Aug 30 14:45:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 14:45:06.513390 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44574dad-1a66-4e7c-89af-87163aa32984 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:06.513686 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44574dad-1a66-4e7c-89af-87163aa32984 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:06.513960 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44574dad-1a66-4e7c-89af-87163aa32984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:06.538972 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-44574dad-1a66-4e7c-89af-87163aa32984 None None] MFA Rules not processed for user `8878de66a03044e38e507028853a0cd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:06.540351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-44574dad-1a66-4e7c-89af-87163aa32984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:06.550903 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1184/2388] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Aug 30 14:45:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:06.562550 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c95387-f481-470e-81b7-3579d73fe083 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:06.575202 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c95387-f481-470e-81b7-3579d73fe083 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:06.579218 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a7ca5e-6a4c-4560-a2ee-3c5970ebd8db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:06.579867 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7ca5e-6a4c-4560-a2ee-3c5970ebd8db service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:06.580173 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7ca5e-6a4c-4560-a2ee-3c5970ebd8db service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:06.580444 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7ca5e-6a4c-4560-a2ee-3c5970ebd8db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:06.592806 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a7ca5e-6a4c-4560-a2ee-3c5970ebd8db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:06.594303 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a7ca5e-6a4c-4560-a2ee-3c5970ebd8db service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:06.607084 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1205/2389] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:45:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:45:06.719670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a7b50-ea62-4fa8-9abd-3063f9329d6c None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:06.720025 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a7b50-ea62-4fa8-9abd-3063f9329d6c None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:06.720511 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a7b50-ea62-4fa8-9abd-3063f9329d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:06.747007 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-412a7b50-ea62-4fa8-9abd-3063f9329d6c None None] MFA Rules not processed for user `f60bee471e954932a928ab2cf7d66c82`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:06.748578 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-412a7b50-ea62-4fa8-9abd-3063f9329d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:06.762032 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1185/2390] 10.222.0.30 () {58 vars in 967 bytes} [Tue Aug 30 14:45:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:06.775003 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a7ca5e-6a4c-4560-a2ee-3c5970ebd8db service neutron] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:06.788697 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a7ca5e-6a4c-4560-a2ee-3c5970ebd8db service neutron] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:06.795535 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e104df83-eedf-4102-86fb-4f341825675b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a18b13e3708144e1b5e8df80ec3a4e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:06.796153 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104df83-eedf-4102-86fb-4f341825675b service swift] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:06.796433 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104df83-eedf-4102-86fb-4f341825675b service swift] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:06.796711 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104df83-eedf-4102-86fb-4f341825675b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:06.806402 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e104df83-eedf-4102-86fb-4f341825675b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:06.808737 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e104df83-eedf-4102-86fb-4f341825675b service swift] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:06.812867 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:06.820317 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1206/2391] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Aug 30 14:45:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 30 14:45:07.111028 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae65312c-f4e8-4005-b66c-3bedcfa6802e None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:07.111557 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae65312c-f4e8-4005-b66c-3bedcfa6802e None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:07.111673 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae65312c-f4e8-4005-b66c-3bedcfa6802e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:07.134004 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-ae65312c-f4e8-4005-b66c-3bedcfa6802e None None] MFA Rules not processed for user `f60bee471e954932a928ab2cf7d66c82`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:07.135485 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-ae65312c-f4e8-4005-b66c-3bedcfa6802e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:07.145233 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1186/2392] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Aug 30 14:45:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:45:07.159101 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e104df83-eedf-4102-86fb-4f341825675b service swift] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:07.171361 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e104df83-eedf-4102-86fb-4f341825675b service swift] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:07.178046 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2343ae77-82ac-47d5-9e47-102d55f6fdd3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a18b13e3708144e1b5e8df80ec3a4e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:07.179005 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343ae77-82ac-47d5-9e47-102d55f6fdd3 service swift] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:07.179320 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343ae77-82ac-47d5-9e47-102d55f6fdd3 service swift] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:07.179594 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343ae77-82ac-47d5-9e47-102d55f6fdd3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:07.189489 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2343ae77-82ac-47d5-9e47-102d55f6fdd3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:07.191385 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2343ae77-82ac-47d5-9e47-102d55f6fdd3 service swift] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:07.202430 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1207/2393] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Aug 30 14:45:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 14:45:07.698243 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4381a157-f07e-4815-aeec-e65151ee3882 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:07.698534 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4381a157-f07e-4815-aeec-e65151ee3882 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:07.698816 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4381a157-f07e-4815-aeec-e65151ee3882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:07.720575 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-4381a157-f07e-4815-aeec-e65151ee3882 None None] MFA Rules not processed for user `f60bee471e954932a928ab2cf7d66c82`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:07.721859 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-4381a157-f07e-4815-aeec-e65151ee3882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:07.729155 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1187/2394] 10.222.0.30 () {58 vars in 967 bytes} [Tue Aug 30 14:45:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:07.741973 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2343ae77-82ac-47d5-9e47-102d55f6fdd3 service swift] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:07.753072 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2343ae77-82ac-47d5-9e47-102d55f6fdd3 service swift] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:07.758285 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79caa85d-2d6f-48d6-a4bc-c89d1f71e9a4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a18b13e3708144e1b5e8df80ec3a4e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:07.758995 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79caa85d-2d6f-48d6-a4bc-c89d1f71e9a4 service swift] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:07.759164 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79caa85d-2d6f-48d6-a4bc-c89d1f71e9a4 service swift] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:07.759536 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79caa85d-2d6f-48d6-a4bc-c89d1f71e9a4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:07.768724 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79caa85d-2d6f-48d6-a4bc-c89d1f71e9a4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:07.770287 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79caa85d-2d6f-48d6-a4bc-c89d1f71e9a4 service swift] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:07.781561 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1208/2395] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Aug 30 14:45:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 14:45:10.438273 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4381a157-f07e-4815-aeec-e65151ee3882 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:10.439841 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-4381a157-f07e-4815-aeec-e65151ee3882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:10.464906 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4381a157-f07e-4815-aeec-e65151ee3882 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:10.470450 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd09e674-c59b-4573-b510-ee4635b6f2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:10.471378 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd09e674-c59b-4573-b510-ee4635b6f2af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:10.471842 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd09e674-c59b-4573-b510-ee4635b6f2af admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:10.472335 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd09e674-c59b-4573-b510-ee4635b6f2af admin admin] PATH_INFO: `/v3/users/08b4f8865dd14290ab714039a2e424fb` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:10.473660 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd09e674-c59b-4573-b510-ee4635b6f2af admin admin] RBAC: Authorizing `identity:delete_user(user_id=08b4f8865dd14290ab714039a2e424fb)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:10.476100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd09e674-c59b-4573-b510-ee4635b6f2af admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:10.547797 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:10.567967 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-bd09e674-c59b-4573-b510-ee4635b6f2af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08b4f8865dd14290ab714039a2e424fb'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:10.568743 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-bd09e674-c59b-4573-b510-ee4635b6f2af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08b4f8865dd14290ab714039a2e424fb'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:10.590357 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-bd09e674-c59b-4573-b510-ee4635b6f2af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08b4f8865dd14290ab714039a2e424fb'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:10.600059 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-bd09e674-c59b-4573-b510-ee4635b6f2af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08b4f8865dd14290ab714039a2e424fb'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:10.608637 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-bd09e674-c59b-4573-b510-ee4635b6f2af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08b4f8865dd14290ab714039a2e424fb'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:10.619954 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1188/2396] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:45:10 2022] DELETE /identity/v3/users/08b4f8865dd14290ab714039a2e424fb => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:10.621891 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509d300-ba10-416b-863a-60421b798bd2 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:10.622905 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509d300-ba10-416b-863a-60421b798bd2 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:10.623265 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509d300-ba10-416b-863a-60421b798bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:10.649691 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-4509d300-ba10-416b-863a-60421b798bd2 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:10.653130 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-4509d300-ba10-416b-863a-60421b798bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:10.738503 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1209/2397] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:45:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:10.748660 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd09e674-c59b-4573-b510-ee4635b6f2af admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:10.750204 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-bd09e674-c59b-4573-b510-ee4635b6f2af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:10.814231 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd09e674-c59b-4573-b510-ee4635b6f2af admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:10.818169 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f72f54-9247-44fc-bf61-a7c337210201 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:10.818773 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f72f54-9247-44fc-bf61-a7c337210201 service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:10.819055 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f72f54-9247-44fc-bf61-a7c337210201 service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:10.819387 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f72f54-9247-44fc-bf61-a7c337210201 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:10.829019 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f72f54-9247-44fc-bf61-a7c337210201 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:10.830435 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f72f54-9247-44fc-bf61-a7c337210201 service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:10.839787 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1189/2398] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:45:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:45:11.060933 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4509d300-ba10-416b-863a-60421b798bd2 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:11.062786 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-4509d300-ba10-416b-863a-60421b798bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:11.074950 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4509d300-ba10-416b-863a-60421b798bd2 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:11.079952 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dae75a-5331-4acb-b636-ac8f40eb8141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:11.080588 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dae75a-5331-4acb-b636-ac8f40eb8141 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:11.080866 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dae75a-5331-4acb-b636-ac8f40eb8141 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:11.081137 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dae75a-5331-4acb-b636-ac8f40eb8141 admin admin] PATH_INFO: `/v3/projects/fe47f61a4dbf4e669ff257d299e92ee4` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:11.082363 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dae75a-5331-4acb-b636-ac8f40eb8141 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe47f61a4dbf4e669ff257d299e92ee4)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:11.084373 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dae75a-5331-4acb-b636-ac8f40eb8141 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:11.145724 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-10dae75a-5331-4acb-b636-ac8f40eb8141 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe47f61a4dbf4e669ff257d299e92ee4', 'request_id': 'req-10dae75a-5331-4acb-b636-ac8f40eb8141'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:11.151667 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-10dae75a-5331-4acb-b636-ac8f40eb8141 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe47f61a4dbf4e669ff257d299e92ee4', 'request_id': 'req-10dae75a-5331-4acb-b636-ac8f40eb8141'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:11.158269 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-10dae75a-5331-4acb-b636-ac8f40eb8141 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe47f61a4dbf4e669ff257d299e92ee4', 'request_id': 'req-10dae75a-5331-4acb-b636-ac8f40eb8141'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:11.170794 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-10dae75a-5331-4acb-b636-ac8f40eb8141 admin admin] The token cache is being invalidate because project fe47f61a4dbf4e669ff257d299e92ee4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:45:11.171080 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-10dae75a-5331-4acb-b636-ac8f40eb8141 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:11.171760 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-10dae75a-5331-4acb-b636-ac8f40eb8141 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:11.173231 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1210/2399] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:45:11 2022] DELETE /identity/v3/projects/fe47f61a4dbf4e669ff257d299e92ee4 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:11.178036 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f2a80d-900b-4008-a67c-5fa31d14b1e3 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:11.178322 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f2a80d-900b-4008-a67c-5fa31d14b1e3 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:11.178595 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f2a80d-900b-4008-a67c-5fa31d14b1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:11.199118 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-85f2a80d-900b-4008-a67c-5fa31d14b1e3 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:11.200339 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-85f2a80d-900b-4008-a67c-5fa31d14b1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:11.261897 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1190/2400] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:45:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:11.269659 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e31ee8-6610-45c3-932b-6c7329a132cb None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:11.269946 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e31ee8-6610-45c3-932b-6c7329a132cb None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:11.270219 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e31ee8-6610-45c3-932b-6c7329a132cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:11.289738 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-23e31ee8-6610-45c3-932b-6c7329a132cb None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:11.291817 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-23e31ee8-6610-45c3-932b-6c7329a132cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:11.299697 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1211/2401] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:45:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:11.303960 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f2a80d-900b-4008-a67c-5fa31d14b1e3 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:11.314150 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f2a80d-900b-4008-a67c-5fa31d14b1e3 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:11.318975 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb554c-2a4f-449c-8f3e-22bad3468567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:11.319602 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb554c-2a4f-449c-8f3e-22bad3468567 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:11.319882 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb554c-2a4f-449c-8f3e-22bad3468567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:11.320153 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb554c-2a4f-449c-8f3e-22bad3468567 admin admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:11.320793 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fb554c-2a4f-449c-8f3e-22bad3468567 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:45:11.321095 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fb554c-2a4f-449c-8f3e-22bad3468567 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:11.322718 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fb554c-2a4f-449c-8f3e-22bad3468567 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:11.334137 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1191/2402] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 14:45:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:45:11.338529 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e31ee8-6610-45c3-932b-6c7329a132cb None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:11.347752 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e31ee8-6610-45c3-932b-6c7329a132cb None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:11.358220 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0576e773-42a9-4982-bbd6-5518f98e5908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:11.358834 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0576e773-42a9-4982-bbd6-5518f98e5908 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:11.359114 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0576e773-42a9-4982-bbd6-5518f98e5908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:11.359423 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0576e773-42a9-4982-bbd6-5518f98e5908 admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:11.360070 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0576e773-42a9-4982-bbd6-5518f98e5908 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:11.361964 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0576e773-42a9-4982-bbd6-5518f98e5908 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:11.389339 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1212/2403] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:45:11 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:45:11.393675 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb554c-2a4f-449c-8f3e-22bad3468567 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:11.403892 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb554c-2a4f-449c-8f3e-22bad3468567 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:11.449712 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:11.464607 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d903c6-5cad-48dd-8ae2-231ad8484940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:11.465205 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d903c6-5cad-48dd-8ae2-231ad8484940 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:11.465481 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d903c6-5cad-48dd-8ae2-231ad8484940 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:11.465750 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d903c6-5cad-48dd-8ae2-231ad8484940 admin admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:11.466375 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d903c6-5cad-48dd-8ae2-231ad8484940 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:11.468320 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d903c6-5cad-48dd-8ae2-231ad8484940 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:11.489474 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1192/2404] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:45:11 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:45:11.493592 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0576e773-42a9-4982-bbd6-5518f98e5908 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:11.504624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0576e773-42a9-4982-bbd6-5518f98e5908 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:11.509165 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca8351-6465-4121-b2f6-ec64fe54dee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:11.509868 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca8351-6465-4121-b2f6-ec64fe54dee1 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:11.510144 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca8351-6465-4121-b2f6-ec64fe54dee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:11.510412 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca8351-6465-4121-b2f6-ec64fe54dee1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:11.511043 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dca8351-6465-4121-b2f6-ec64fe54dee1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:11.512695 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dca8351-6465-4121-b2f6-ec64fe54dee1 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:11.522401 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1213/2405] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:45:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:45:11.526843 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d903c6-5cad-48dd-8ae2-231ad8484940 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:11.536645 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d903c6-5cad-48dd-8ae2-231ad8484940 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:11.541766 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969bce21-02b2-4c9e-9031-3549c3e0c4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:11.542346 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969bce21-02b2-4c9e-9031-3549c3e0c4c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:11.542623 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969bce21-02b2-4c9e-9031-3549c3e0c4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:11.542894 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969bce21-02b2-4c9e-9031-3549c3e0c4c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:11.543560 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969bce21-02b2-4c9e-9031-3549c3e0c4c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:11.545174 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969bce21-02b2-4c9e-9031-3549c3e0c4c2 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:11.555754 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1193/2406] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:45:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:45:11.559732 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca8351-6465-4121-b2f6-ec64fe54dee1 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:11.569894 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca8351-6465-4121-b2f6-ec64fe54dee1 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:11.574542 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3415e-2b02-4939-8873-df7b16d29fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:11.575185 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3415e-2b02-4939-8873-df7b16d29fe8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:11.575496 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3415e-2b02-4939-8873-df7b16d29fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:11.575766 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3415e-2b02-4939-8873-df7b16d29fe8 admin admin] PATH_INFO: `/v3/projects/1bfe11f6c0da4508b94a01afb80c8483/users/6fac2f6b6c124ddf99a888aa0c30f83b/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:11.591082 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:11.594556 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f3415e-2b02-4939-8873-df7b16d29fe8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bfe11f6c0da4508b94a01afb80c8483, user_id=6fac2f6b6c124ddf99a888aa0c30f83b, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:11.597790 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f3415e-2b02-4939-8873-df7b16d29fe8 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:11.613245 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1214/2407] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:45:11 2022] PUT /identity/v3/projects/1bfe11f6c0da4508b94a01afb80c8483/users/6fac2f6b6c124ddf99a888aa0c30f83b/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:11.618225 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0329755e-effd-44df-8186-aacf5e202912 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:11.618782 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0329755e-effd-44df-8186-aacf5e202912 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:11.619150 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0329755e-effd-44df-8186-aacf5e202912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:11.638514 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-0329755e-effd-44df-8186-aacf5e202912 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:11.639749 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-0329755e-effd-44df-8186-aacf5e202912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:11.705120 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1194/2408] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:45:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:11.715927 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3415e-2b02-4939-8873-df7b16d29fe8 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:11.718282 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-a3f3415e-2b02-4939-8873-df7b16d29fe8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:11.779711 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3415e-2b02-4939-8873-df7b16d29fe8 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:11.783368 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff8dcb9-59ed-4b64-92fd-18440a8e2052 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:11.783968 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8dcb9-59ed-4b64-92fd-18440a8e2052 service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:11.784244 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8dcb9-59ed-4b64-92fd-18440a8e2052 service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:11.784512 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8dcb9-59ed-4b64-92fd-18440a8e2052 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:11.793919 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff8dcb9-59ed-4b64-92fd-18440a8e2052 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:11.795365 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff8dcb9-59ed-4b64-92fd-18440a8e2052 service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:11.804676 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1215/2409] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:45:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:45:12.586094 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c744f054-b7a8-4a1c-8b8e-2c9229a01287 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:12.586636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c744f054-b7a8-4a1c-8b8e-2c9229a01287 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:12.586636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c744f054-b7a8-4a1c-8b8e-2c9229a01287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:12.616265 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-c744f054-b7a8-4a1c-8b8e-2c9229a01287 None None] MFA Rules not processed for user `6fac2f6b6c124ddf99a888aa0c30f83b`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:12.617499 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-c744f054-b7a8-4a1c-8b8e-2c9229a01287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:12.685865 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1195/2410] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:45:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:12.696768 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff8dcb9-59ed-4b64-92fd-18440a8e2052 service neutron] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:12.698411 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-6ff8dcb9-59ed-4b64-92fd-18440a8e2052 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:12.757517 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff8dcb9-59ed-4b64-92fd-18440a8e2052 service neutron] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:12.761217 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803c71bd-5a0b-4564-b5e7-b1503c67167f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:12.761824 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803c71bd-5a0b-4564-b5e7-b1503c67167f service cinder] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:12.762096 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803c71bd-5a0b-4564-b5e7-b1503c67167f service cinder] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:12.762367 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803c71bd-5a0b-4564-b5e7-b1503c67167f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:12.771049 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803c71bd-5a0b-4564-b5e7-b1503c67167f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:12.772505 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803c71bd-5a0b-4564-b5e7-b1503c67167f service cinder] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:12.783429 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1216/2411] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:45:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:45:14.141530 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5328e-71b4-458e-b649-b6eada971833 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:14.142008 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5328e-71b4-458e-b649-b6eada971833 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:14.142096 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5328e-71b4-458e-b649-b6eada971833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:14.163113 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-05b5328e-71b4-458e-b649-b6eada971833 None None] MFA Rules not processed for user `6fac2f6b6c124ddf99a888aa0c30f83b`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:14.164347 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-05b5328e-71b4-458e-b649-b6eada971833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:14.172308 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1196/2412] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:45:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:14.198597 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba6d10-7ae4-4641-83c2-aa15ec0f94b2 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:14.199112 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba6d10-7ae4-4641-83c2-aa15ec0f94b2 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:14.199358 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba6d10-7ae4-4641-83c2-aa15ec0f94b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:14.232470 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-51ba6d10-7ae4-4641-83c2-aa15ec0f94b2 None None] MFA Rules not processed for user `f60bee471e954932a928ab2cf7d66c82`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:14.234218 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-51ba6d10-7ae4-4641-83c2-aa15ec0f94b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:14.308920 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1217/2413] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Aug 30 14:45:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:14.322401 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b5328e-71b4-458e-b649-b6eada971833 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:14.323936 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-05b5328e-71b4-458e-b649-b6eada971833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:14.389499 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:14.420776 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b5328e-71b4-458e-b649-b6eada971833 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:14.425898 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c3519b-b182-4700-935e-82e1a8375322 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a18b13e3708144e1b5e8df80ec3a4e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:14.426515 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c3519b-b182-4700-935e-82e1a8375322 service swift] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:14.426795 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c3519b-b182-4700-935e-82e1a8375322 service swift] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:14.427067 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c3519b-b182-4700-935e-82e1a8375322 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:14.436299 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c3519b-b182-4700-935e-82e1a8375322 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:14.437999 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c3519b-b182-4700-935e-82e1a8375322 service swift] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:14.446499 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1197/2414] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Aug 30 14:45:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 14:45:19.694885 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ba6d10-7ae4-4641-83c2-aa15ec0f94b2 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:19.696368 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-51ba6d10-7ae4-4641-83c2-aa15ec0f94b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:19.711120 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ba6d10-7ae4-4641-83c2-aa15ec0f94b2 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:19.716712 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf7059-2991-482b-a72f-411771ce9793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:19.717380 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf7059-2991-482b-a72f-411771ce9793 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:19.717660 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf7059-2991-482b-a72f-411771ce9793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:19.717934 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf7059-2991-482b-a72f-411771ce9793 admin admin] PATH_INFO: `/v3/users/17ac76bb8f2543409fc2a64541d591a4` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:19.719116 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adf7059-2991-482b-a72f-411771ce9793 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17ac76bb8f2543409fc2a64541d591a4)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:19.721195 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adf7059-2991-482b-a72f-411771ce9793 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:19.805122 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:19.816600 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-3adf7059-2991-482b-a72f-411771ce9793 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17ac76bb8f2543409fc2a64541d591a4'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:19.817609 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-3adf7059-2991-482b-a72f-411771ce9793 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17ac76bb8f2543409fc2a64541d591a4'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:19.826188 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-3adf7059-2991-482b-a72f-411771ce9793 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17ac76bb8f2543409fc2a64541d591a4'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:19.835557 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-3adf7059-2991-482b-a72f-411771ce9793 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17ac76bb8f2543409fc2a64541d591a4'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:19.836413 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-3adf7059-2991-482b-a72f-411771ce9793 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17ac76bb8f2543409fc2a64541d591a4'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:19.856393 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1218/2415] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:45:19 2022] DELETE /identity/v3/users/17ac76bb8f2543409fc2a64541d591a4 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:19.860703 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a85a2fe-e103-4dd8-a00d-643cb93d7a1d None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:19.861209 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a85a2fe-e103-4dd8-a00d-643cb93d7a1d None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:19.861702 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a85a2fe-e103-4dd8-a00d-643cb93d7a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:19.902095 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-6a85a2fe-e103-4dd8-a00d-643cb93d7a1d None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:19.903528 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-6a85a2fe-e103-4dd8-a00d-643cb93d7a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:19.969114 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1198/2416] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:45:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:45:19.977117 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf7059-2991-482b-a72f-411771ce9793 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:19.980028 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-3adf7059-2991-482b-a72f-411771ce9793 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:20.081387 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf7059-2991-482b-a72f-411771ce9793 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:20.085303 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae2ed6-f65d-4c9b-9335-79021cd1caca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:20.085987 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae2ed6-f65d-4c9b-9335-79021cd1caca service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:20.086336 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae2ed6-f65d-4c9b-9335-79021cd1caca service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:20.086679 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae2ed6-f65d-4c9b-9335-79021cd1caca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:20.097799 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fae2ed6-f65d-4c9b-9335-79021cd1caca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:20.099432 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fae2ed6-f65d-4c9b-9335-79021cd1caca service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:20.114628 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1219/2417] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:45:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:45:20.425413 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a85a2fe-e103-4dd8-a00d-643cb93d7a1d None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:20.426937 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-6a85a2fe-e103-4dd8-a00d-643cb93d7a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:20.441487 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a85a2fe-e103-4dd8-a00d-643cb93d7a1d None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:20.446238 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74666327-eeb3-4c24-a704-11899582089a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:20.447975 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74666327-eeb3-4c24-a704-11899582089a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:20.448365 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74666327-eeb3-4c24-a704-11899582089a admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:20.448641 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74666327-eeb3-4c24-a704-11899582089a admin admin] PATH_INFO: `/v3/projects/92ffb73db5f54a6d8d5d1f1e50103d22` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:20.449827 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74666327-eeb3-4c24-a704-11899582089a admin admin] RBAC: Authorizing `identity:delete_project(project_id=92ffb73db5f54a6d8d5d1f1e50103d22)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:20.451780 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74666327-eeb3-4c24-a704-11899582089a admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:20.514384 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-74666327-eeb3-4c24-a704-11899582089a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92ffb73db5f54a6d8d5d1f1e50103d22', 'request_id': 'req-74666327-eeb3-4c24-a704-11899582089a'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:20.521573 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-74666327-eeb3-4c24-a704-11899582089a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92ffb73db5f54a6d8d5d1f1e50103d22', 'request_id': 'req-74666327-eeb3-4c24-a704-11899582089a'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:20.529849 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-74666327-eeb3-4c24-a704-11899582089a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92ffb73db5f54a6d8d5d1f1e50103d22', 'request_id': 'req-74666327-eeb3-4c24-a704-11899582089a'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:20.550916 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-74666327-eeb3-4c24-a704-11899582089a admin admin] The token cache is being invalidate because project 92ffb73db5f54a6d8d5d1f1e50103d22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:45:20.551463 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-74666327-eeb3-4c24-a704-11899582089a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:20.552305 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-74666327-eeb3-4c24-a704-11899582089a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:45:20.554121 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1199/2418] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:45:20 2022] DELETE /identity/v3/projects/92ffb73db5f54a6d8d5d1f1e50103d22 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:45:28.871747 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae2ed6-f65d-4c9b-9335-79021cd1caca service neutron] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:28.873175 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-3fae2ed6-f65d-4c9b-9335-79021cd1caca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:28.945718 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae2ed6-f65d-4c9b-9335-79021cd1caca service neutron] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:29.000112 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:29.010497 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac8c889-1a38-4361-ae88-8d70e0a82115 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcf302a170934c25b39699088dac326a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:29.011409 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac8c889-1a38-4361-ae88-8d70e0a82115 service placement] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:29.011922 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac8c889-1a38-4361-ae88-8d70e0a82115 service placement] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:29.012250 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac8c889-1a38-4361-ae88-8d70e0a82115 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:29.014349 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-2ac8c889-1a38-4361-ae88-8d70e0a82115 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:29.026164 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac8c889-1a38-4361-ae88-8d70e0a82115 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:29.027803 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac8c889-1a38-4361-ae88-8d70e0a82115 service placement] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:29.038348 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1220/2419] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:45:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:45:30.543924 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=64,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:39.034114 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=5 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:40.740353 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2178b4d4-a852-412a-91e5-41ab93687e3e None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:40.740874 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2178b4d4-a852-412a-91e5-41ab93687e3e None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:40.740980 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2178b4d4-a852-412a-91e5-41ab93687e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:40.761001 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-2178b4d4-a852-412a-91e5-41ab93687e3e None None] MFA Rules not processed for user `a5dd8cf0a16345c5b2973552f0881893`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:40.762212 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-2178b4d4-a852-412a-91e5-41ab93687e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:40.816898 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1200/2420] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:45:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:43.233391 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e40c9-7995-4419-b8d9-15f287095107 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:43.233899 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e40c9-7995-4419-b8d9-15f287095107 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:43.234124 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e40c9-7995-4419-b8d9-15f287095107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:43.254757 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-6d7e40c9-7995-4419-b8d9-15f287095107 None None] MFA Rules not processed for user `f60bee471e954932a928ab2cf7d66c82`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:45:43.256038 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-6d7e40c9-7995-4419-b8d9-15f287095107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:43.314136 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1221/2421] 10.222.0.30 () {58 vars in 967 bytes} [Tue Aug 30 14:45:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:45:43.328865 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2178b4d4-a852-412a-91e5-41ab93687e3e None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:45:43.330215 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-2178b4d4-a852-412a-91e5-41ab93687e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:45:43.403802 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2178b4d4-a852-412a-91e5-41ab93687e3e None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:45:43.408669 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6359240e-bce8-4c31-882c-69304b947a65 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a18b13e3708144e1b5e8df80ec3a4e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:45:43.409265 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359240e-bce8-4c31-882c-69304b947a65 service swift] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:45:43.409549 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359240e-bce8-4c31-882c-69304b947a65 service swift] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:45:43.409919 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359240e-bce8-4c31-882c-69304b947a65 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:45:43.418566 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6359240e-bce8-4c31-882c-69304b947a65 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:45:43.420205 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6359240e-bce8-4c31-882c-69304b947a65 service swift] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:45:43.428291 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1201/2422] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Aug 30 14:45:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 14:45:53.307478 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=25 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:45:53.425028 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=57 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:46:06.951848 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0952de1-9f4f-48bd-9851-bbd434827758 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:06.952390 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0952de1-9f4f-48bd-9851-bbd434827758 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:06.952724 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0952de1-9f4f-48bd-9851-bbd434827758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:06.981210 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-d0952de1-9f4f-48bd-9851-bbd434827758 None None] MFA Rules not processed for user `f60bee471e954932a928ab2cf7d66c82`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:46:06.982923 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-d0952de1-9f4f-48bd-9851-bbd434827758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:46:06.994680 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1222/2423] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Aug 30 14:46:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:46:07.007079 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6359240e-bce8-4c31-882c-69304b947a65 service swift] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:46:07.017836 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6359240e-bce8-4c31-882c-69304b947a65 service swift] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:46:07.023623 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb8aa21-a9c3-4485-bd1d-457476a7e6e8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a18b13e3708144e1b5e8df80ec3a4e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:46:07.024257 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb8aa21-a9c3-4485-bd1d-457476a7e6e8 service swift] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:46:07.024533 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb8aa21-a9c3-4485-bd1d-457476a7e6e8 service swift] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:46:07.024834 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb8aa21-a9c3-4485-bd1d-457476a7e6e8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:46:07.033269 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb8aa21-a9c3-4485-bd1d-457476a7e6e8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:46:07.034914 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb8aa21-a9c3-4485-bd1d-457476a7e6e8 service swift] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:46:07.043807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1202/2424] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Aug 30 14:46:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 14:46:16.973432 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=3 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:46:17.041023 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=6 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:47:13.124700 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0952de1-9f4f-48bd-9851-bbd434827758 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:13.126095 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-d0952de1-9f4f-48bd-9851-bbd434827758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:13.181480 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0952de1-9f4f-48bd-9851-bbd434827758 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:13.185839 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86241694-fc0f-4c97-8f37-cb64d118c377 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:13.186432 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86241694-fc0f-4c97-8f37-cb64d118c377 service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:13.186705 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86241694-fc0f-4c97-8f37-cb64d118c377 service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:13.186971 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86241694-fc0f-4c97-8f37-cb64d118c377 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:13.188680 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-86241694-fc0f-4c97-8f37-cb64d118c377 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:13.257614 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86241694-fc0f-4c97-8f37-cb64d118c377 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:13.259038 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86241694-fc0f-4c97-8f37-cb64d118c377 service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:13.270037 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1223/2425] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:47:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:47:15.793410 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb8aa21-a9c3-4485-bd1d-457476a7e6e8 service swift] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:15.794986 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-5cb8aa21-a9c3-4485-bd1d-457476a7e6e8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:15.855679 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb8aa21-a9c3-4485-bd1d-457476a7e6e8 service swift] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:15.859386 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ff1c6-756c-4519-8093-099c1c55cc95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:15.860005 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ff1c6-756c-4519-8093-099c1c55cc95 service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:15.860282 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ff1c6-756c-4519-8093-099c1c55cc95 service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:15.860550 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ff1c6-756c-4519-8093-099c1c55cc95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:15.862242 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-ec6ff1c6-756c-4519-8093-099c1c55cc95 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:15.876009 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6ff1c6-756c-4519-8093-099c1c55cc95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:15.877447 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6ff1c6-756c-4519-8093-099c1c55cc95 service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:15.888813 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1203/2426] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:47:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:47:22.349257 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58436d24-d271-489c-a033-489be52e3e6a None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:22.349761 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58436d24-d271-489c-a033-489be52e3e6a None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:22.349878 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58436d24-d271-489c-a033-489be52e3e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:22.370035 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-58436d24-d271-489c-a033-489be52e3e6a None None] MFA Rules not processed for user `ca9bafc5ff4e43719f37e62cad946cf7`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:47:22.371279 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-58436d24-d271-489c-a033-489be52e3e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:22.426174 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1224/2427] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:47:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:47:25.883505 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=26 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:47:30.564340 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ff1c6-756c-4519-8093-099c1c55cc95 service cinder] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:30.565777 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-ec6ff1c6-756c-4519-8093-099c1c55cc95 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:30.580161 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ff1c6-756c-4519-8093-099c1c55cc95 service cinder] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:30.591814 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e2e897-c2db-41f7-8c4f-774662e34f97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:30.592426 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2e897-c2db-41f7-8c4f-774662e34f97 service nova] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:30.592705 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2e897-c2db-41f7-8c4f-774662e34f97 service nova] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:30.592975 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2e897-c2db-41f7-8c4f-774662e34f97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:30.594732 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-97e2e897-c2db-41f7-8c4f-774662e34f97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:30.661840 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e2e897-c2db-41f7-8c4f-774662e34f97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:30.663381 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e2e897-c2db-41f7-8c4f-774662e34f97 service nova] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:30.672333 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1204/2428] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:47:30 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:47:30.855707 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20a150-41b3-4407-abba-a2eaff1d7d18 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:30.856314 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20a150-41b3-4407-abba-a2eaff1d7d18 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:30.856314 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20a150-41b3-4407-abba-a2eaff1d7d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:30.875995 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-2b20a150-41b3-4407-abba-a2eaff1d7d18 None None] MFA Rules not processed for user `35f28006916040c3b988de6c15093166`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:47:30.877234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-2b20a150-41b3-4407-abba-a2eaff1d7d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:30.888904 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1225/2429] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:47:30 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 14:47:32.492157 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e2e897-c2db-41f7-8c4f-774662e34f97 service nova] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:32.494209 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-97e2e897-c2db-41f7-8c4f-774662e34f97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:32.509340 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e2e897-c2db-41f7-8c4f-774662e34f97 service nova] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:32.517222 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0ac3a-d876-4de5-ad40-219470e9108a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:32.518120 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0ac3a-d876-4de5-ad40-219470e9108a service nova] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:32.518594 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0ac3a-d876-4de5-ad40-219470e9108a service nova] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:32.519056 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0ac3a-d876-4de5-ad40-219470e9108a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:32.521148 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-57e0ac3a-d876-4de5-ad40-219470e9108a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:32.533986 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e0ac3a-d876-4de5-ad40-219470e9108a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:32.535724 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e0ac3a-d876-4de5-ad40-219470e9108a service nova] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:32.547981 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1205/2430] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:47:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:47:35.705485 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b20a150-41b3-4407-abba-a2eaff1d7d18 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:47:35.721295 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b20a150-41b3-4407-abba-a2eaff1d7d18 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:47:35.726906 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a7701f-6db7-44a8-b9a1-3a40310ec10e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:47:35.727556 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7701f-6db7-44a8-b9a1-3a40310ec10e service nova] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:47:35.727948 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7701f-6db7-44a8-b9a1-3a40310ec10e service nova] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:47:35.728228 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7701f-6db7-44a8-b9a1-3a40310ec10e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:47:35.730369 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-64a7701f-6db7-44a8-b9a1-3a40310ec10e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:47:35.779566 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:47:35.813256 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a7701f-6db7-44a8-b9a1-3a40310ec10e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:47:35.814692 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a7701f-6db7-44a8-b9a1-3a40310ec10e service nova] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:47:35.828188 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1226/2431] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:47:35 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:47:42.542051 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=34 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:47:45.822220 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=10 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:48:18.887741 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0ac3a-d876-4de5-ad40-219470e9108a service nova] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:48:18.897244 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0ac3a-d876-4de5-ad40-219470e9108a service nova] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:48:18.901528 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7943411-64dc-4ea9-8848-8441b7859bbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:48:18.902118 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7943411-64dc-4ea9-8848-8441b7859bbf service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:48:18.902399 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7943411-64dc-4ea9-8848-8441b7859bbf service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:48:18.902675 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7943411-64dc-4ea9-8848-8441b7859bbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:48:18.904436 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-b7943411-64dc-4ea9-8848-8441b7859bbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:48:18.916705 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7943411-64dc-4ea9-8848-8441b7859bbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:48:18.918131 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7943411-64dc-4ea9-8848-8441b7859bbf service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:48:18.926090 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1206/2432] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:48:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:48:28.922544 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=6 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:03.145774 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a7701f-6db7-44a8-b9a1-3a40310ec10e service nova] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:50:03.147242 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-64a7701f-6db7-44a8-b9a1-3a40310ec10e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:03.158856 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a7701f-6db7-44a8-b9a1-3a40310ec10e service nova] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:50:03.162414 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eefa41-e01d-420b-978e-bb4784afbf8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:50:03.163795 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eefa41-e01d-420b-978e-bb4784afbf8e service cinder] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:03.164073 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eefa41-e01d-420b-978e-bb4784afbf8e service cinder] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:03.164343 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eefa41-e01d-420b-978e-bb4784afbf8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:03.166010 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-59eefa41-e01d-420b-978e-bb4784afbf8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:03.176445 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eefa41-e01d-420b-978e-bb4784afbf8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:50:03.177851 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eefa41-e01d-420b-978e-bb4784afbf8e service cinder] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:50:03.187254 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1227/2433] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:50:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:50:12.695854 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7943411-64dc-4ea9-8848-8441b7859bbf service neutron] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:50:12.705097 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7943411-64dc-4ea9-8848-8441b7859bbf service neutron] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:50:12.708688 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f03f6d3-b6fb-4215-8aa7-3e105a1be326 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:50:12.709281 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f03f6d3-b6fb-4215-8aa7-3e105a1be326 service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:12.709592 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f03f6d3-b6fb-4215-8aa7-3e105a1be326 service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:12.709871 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f03f6d3-b6fb-4215-8aa7-3e105a1be326 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:12.711627 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-8f03f6d3-b6fb-4215-8aa7-3e105a1be326 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:12.769318 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f03f6d3-b6fb-4215-8aa7-3e105a1be326 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:50:12.770743 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f03f6d3-b6fb-4215-8aa7-3e105a1be326 service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:50:12.779807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1207/2434] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:50:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:50:13.183281 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=6 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:22.775807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=28 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:50:32.614004 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eefa41-e01d-420b-978e-bb4784afbf8e service cinder] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:50:32.615499 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-59eefa41-e01d-420b-978e-bb4784afbf8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:50:32.706357 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eefa41-e01d-420b-978e-bb4784afbf8e service cinder] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:50:32.711802 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740a24a1-b196-4ec9-9c32-279cb21d74e5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcf302a170934c25b39699088dac326a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:50:32.713038 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a24a1-b196-4ec9-9c32-279cb21d74e5 service placement] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:50:32.713319 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a24a1-b196-4ec9-9c32-279cb21d74e5 service placement] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:50:32.713587 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a24a1-b196-4ec9-9c32-279cb21d74e5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:50:32.722424 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740a24a1-b196-4ec9-9c32-279cb21d74e5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:50:32.724033 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740a24a1-b196-4ec9-9c32-279cb21d74e5 service placement] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:50:32.733773 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1228/2435] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:50:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:50:42.729607 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=37 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:52:16.951562 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f03f6d3-b6fb-4215-8aa7-3e105a1be326 service cinder] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:52:16.983806 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f03f6d3-b6fb-4215-8aa7-3e105a1be326 service cinder] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:52:16.991691 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294eaad4-ecc8-4af1-be44-dd57f1da84f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:52:16.992562 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294eaad4-ecc8-4af1-be44-dd57f1da84f5 service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:52:16.992842 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294eaad4-ecc8-4af1-be44-dd57f1da84f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:52:16.993115 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294eaad4-ecc8-4af1-be44-dd57f1da84f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:52:17.012973 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294eaad4-ecc8-4af1-be44-dd57f1da84f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:52:17.014463 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294eaad4-ecc8-4af1-be44-dd57f1da84f5 service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:52:17.039282 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1208/2436] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:52:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:52:23.231802 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be89e49e-bb11-4efa-934f-7847e89413ef None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:52:23.232255 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be89e49e-bb11-4efa-934f-7847e89413ef None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:52:23.232255 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be89e49e-bb11-4efa-934f-7847e89413ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:52:23.261136 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-be89e49e-bb11-4efa-934f-7847e89413ef None None] MFA Rules not processed for user `f60bee471e954932a928ab2cf7d66c82`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:52:23.262335 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-be89e49e-bb11-4efa-934f-7847e89413ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:52:23.269441 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1229/2437] 10.222.0.30 () {58 vars in 967 bytes} [Tue Aug 30 14:52:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:52:23.316266 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294eaad4-ecc8-4af1-be44-dd57f1da84f5 service cinder] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:52:23.327680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294eaad4-ecc8-4af1-be44-dd57f1da84f5 service cinder] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:52:23.332806 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb4e7ae-25b7-48e9-9f06-c0931235d9dc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a18b13e3708144e1b5e8df80ec3a4e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:52:23.333394 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4e7ae-25b7-48e9-9f06-c0931235d9dc service swift] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:52:23.333675 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4e7ae-25b7-48e9-9f06-c0931235d9dc service swift] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:52:23.333946 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4e7ae-25b7-48e9-9f06-c0931235d9dc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:52:23.342395 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb4e7ae-25b7-48e9-9f06-c0931235d9dc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:52:23.344018 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb4e7ae-25b7-48e9-9f06-c0931235d9dc service swift] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:52:23.352478 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1209/2438] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Aug 30 14:52:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 14:52:23.738665 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be89e49e-bb11-4efa-934f-7847e89413ef None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:52:23.740029 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-be89e49e-bb11-4efa-934f-7847e89413ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:52:23.774810 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be89e49e-bb11-4efa-934f-7847e89413ef None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:52:23.778398 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fbc7cd-a162-4051-ae07-156b983c120a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:52:23.778990 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fbc7cd-a162-4051-ae07-156b983c120a service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:52:23.779296 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fbc7cd-a162-4051-ae07-156b983c120a service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:52:23.779567 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fbc7cd-a162-4051-ae07-156b983c120a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:52:23.781222 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-76fbc7cd-a162-4051-ae07-156b983c120a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:52:23.865271 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fbc7cd-a162-4051-ae07-156b983c120a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:52:23.867191 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fbc7cd-a162-4051-ae07-156b983c120a service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:52:23.886521 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1230/2439] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:52:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:52:24.342313 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb4e7ae-25b7-48e9-9f06-c0931235d9dc service swift] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:52:24.354554 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb4e7ae-25b7-48e9-9f06-c0931235d9dc service swift] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:52:24.359655 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137cabfd-9dea-4b41-99d5-c72309fb0245 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcf302a170934c25b39699088dac326a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:52:24.360255 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137cabfd-9dea-4b41-99d5-c72309fb0245 service placement] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:52:24.360535 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137cabfd-9dea-4b41-99d5-c72309fb0245 service placement] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:52:24.360811 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137cabfd-9dea-4b41-99d5-c72309fb0245 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:52:24.362560 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-137cabfd-9dea-4b41-99d5-c72309fb0245 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:52:24.374946 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137cabfd-9dea-4b41-99d5-c72309fb0245 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:52:24.377154 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137cabfd-9dea-4b41-99d5-c72309fb0245 service placement] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:52:24.391490 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1210/2440] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:52:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:52:24.410569 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fbc7cd-a162-4051-ae07-156b983c120a service neutron] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:52:24.412170 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-76fbc7cd-a162-4051-ae07-156b983c120a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:52:24.426970 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fbc7cd-a162-4051-ae07-156b983c120a service neutron] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:52:24.432905 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c83e08-1ca3-48ce-9a58-36a7afba4396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:52:24.433524 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c83e08-1ca3-48ce-9a58-36a7afba4396 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:52:24.433803 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c83e08-1ca3-48ce-9a58-36a7afba4396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:52:24.434077 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c83e08-1ca3-48ce-9a58-36a7afba4396 admin admin] PATH_INFO: `/v3/users/a5dd8cf0a16345c5b2973552f0881893` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:52:24.435236 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c83e08-1ca3-48ce-9a58-36a7afba4396 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5dd8cf0a16345c5b2973552f0881893)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:52:24.437251 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c83e08-1ca3-48ce-9a58-36a7afba4396 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:52:24.484618 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-38c83e08-1ca3-48ce-9a58-36a7afba4396 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5dd8cf0a16345c5b2973552f0881893'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:52:24.485273 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-38c83e08-1ca3-48ce-9a58-36a7afba4396 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5dd8cf0a16345c5b2973552f0881893'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:52:24.493675 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-38c83e08-1ca3-48ce-9a58-36a7afba4396 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5dd8cf0a16345c5b2973552f0881893'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:52:24.500687 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-38c83e08-1ca3-48ce-9a58-36a7afba4396 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5dd8cf0a16345c5b2973552f0881893'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:52:24.501320 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-38c83e08-1ca3-48ce-9a58-36a7afba4396 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5dd8cf0a16345c5b2973552f0881893'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:52:24.518786 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1231/2441] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:52:24 2022] DELETE /identity/v3/users/a5dd8cf0a16345c5b2973552f0881893 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 14:52:24.702044 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137cabfd-9dea-4b41-99d5-c72309fb0245 service placement] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:52:24.719460 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137cabfd-9dea-4b41-99d5-c72309fb0245 service placement] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:52:24.792269 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cffbce8-07a0-4f29-a553-59d002ea4ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:52:24.792896 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cffbce8-07a0-4f29-a553-59d002ea4ace admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:52:24.793178 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cffbce8-07a0-4f29-a553-59d002ea4ace admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:52:24.793449 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cffbce8-07a0-4f29-a553-59d002ea4ace admin admin] PATH_INFO: `/v3/projects/e67c28ec4d3949dc9d00e70f61b5657f` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:52:24.794561 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cffbce8-07a0-4f29-a553-59d002ea4ace admin admin] RBAC: Authorizing `identity:delete_project(project_id=e67c28ec4d3949dc9d00e70f61b5657f)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:52:24.796530 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cffbce8-07a0-4f29-a553-59d002ea4ace admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:52:24.841930 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-3cffbce8-07a0-4f29-a553-59d002ea4ace admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e67c28ec4d3949dc9d00e70f61b5657f', 'request_id': 'req-3cffbce8-07a0-4f29-a553-59d002ea4ace'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:52:24.847401 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-3cffbce8-07a0-4f29-a553-59d002ea4ace admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e67c28ec4d3949dc9d00e70f61b5657f', 'request_id': 'req-3cffbce8-07a0-4f29-a553-59d002ea4ace'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:52:24.853379 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-3cffbce8-07a0-4f29-a553-59d002ea4ace admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e67c28ec4d3949dc9d00e70f61b5657f', 'request_id': 'req-3cffbce8-07a0-4f29-a553-59d002ea4ace'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:52:24.865533 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-3cffbce8-07a0-4f29-a553-59d002ea4ace admin admin] The token cache is being invalidate because project e67c28ec4d3949dc9d00e70f61b5657f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:52:24.865842 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-3cffbce8-07a0-4f29-a553-59d002ea4ace admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:52:24.867164 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-3cffbce8-07a0-4f29-a553-59d002ea4ace admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:52:24.868683 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1211/2442] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:52:24 2022] DELETE /identity/v3/projects/e67c28ec4d3949dc9d00e70f61b5657f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:52:24.875349 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb02fc59-5404-4d7a-9286-844ccd66ff9a None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:52:24.875699 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb02fc59-5404-4d7a-9286-844ccd66ff9a None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:52:24.876035 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb02fc59-5404-4d7a-9286-844ccd66ff9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:52:24.881019 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:52:24.936014 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-bb02fc59-5404-4d7a-9286-844ccd66ff9a None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:52:24.937210 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-bb02fc59-5404-4d7a-9286-844ccd66ff9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:52:24.996181 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1232/2443] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:52:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:52:25.004001 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee30df2-1109-4d2e-910f-e31b6ad75be6 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:52:25.004282 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee30df2-1109-4d2e-910f-e31b6ad75be6 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:52:25.004549 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee30df2-1109-4d2e-910f-e31b6ad75be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:52:25.025499 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-1ee30df2-1109-4d2e-910f-e31b6ad75be6 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:52:25.026698 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-1ee30df2-1109-4d2e-910f-e31b6ad75be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:52:25.034489 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1212/2444] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:52:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:52:25.039332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb02fc59-5404-4d7a-9286-844ccd66ff9a None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:52:25.048988 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb02fc59-5404-4d7a-9286-844ccd66ff9a None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:52:25.053670 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e2bb33-6915-4912-a741-14e64fe4c304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:52:25.054308 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e2bb33-6915-4912-a741-14e64fe4c304 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:52:25.054589 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e2bb33-6915-4912-a741-14e64fe4c304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:52:25.054860 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e2bb33-6915-4912-a741-14e64fe4c304 admin admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:52:25.055543 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e2bb33-6915-4912-a741-14e64fe4c304 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 14:52:25.055845 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e2bb33-6915-4912-a741-14e64fe4c304 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:52:25.057432 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e2bb33-6915-4912-a741-14e64fe4c304 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:52:25.069080 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1233/2445] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 14:52:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 14:52:25.073776 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee30df2-1109-4d2e-910f-e31b6ad75be6 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:52:25.082918 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee30df2-1109-4d2e-910f-e31b6ad75be6 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:52:25.088491 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728cdecb-96ca-4f61-af80-aeec87013039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:52:25.089088 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728cdecb-96ca-4f61-af80-aeec87013039 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:52:25.089369 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728cdecb-96ca-4f61-af80-aeec87013039 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:52:25.089643 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728cdecb-96ca-4f61-af80-aeec87013039 admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:52:25.090286 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728cdecb-96ca-4f61-af80-aeec87013039 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:52:25.092336 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728cdecb-96ca-4f61-af80-aeec87013039 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:52:25.097355 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:52:25.130052 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1213/2446] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 14:52:25 2022] POST /identity/v3/projects => generated 389 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:52:25.134678 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e2bb33-6915-4912-a741-14e64fe4c304 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:52:25.144837 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e2bb33-6915-4912-a741-14e64fe4c304 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:52:25.201063 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742942b9-ed95-4fb9-a182-8340f03a9ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:52:25.201713 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742942b9-ed95-4fb9-a182-8340f03a9ca7 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:52:25.202021 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742942b9-ed95-4fb9-a182-8340f03a9ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:52:25.202295 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742942b9-ed95-4fb9-a182-8340f03a9ca7 admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:52:25.202928 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742942b9-ed95-4fb9-a182-8340f03a9ca7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:52:25.204932 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742942b9-ed95-4fb9-a182-8340f03a9ca7 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:52:25.228162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1234/2447] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 14:52:25 2022] POST /identity/v3/users => generated 295 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 14:52:25.232272 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728cdecb-96ca-4f61-af80-aeec87013039 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:52:25.242781 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728cdecb-96ca-4f61-af80-aeec87013039 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:52:25.247605 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb4ae56-c61f-4f06-ba3d-fa5b04ba618f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:52:25.248246 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4ae56-c61f-4f06-ba3d-fa5b04ba618f admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:52:25.248561 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4ae56-c61f-4f06-ba3d-fa5b04ba618f admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:52:25.248838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4ae56-c61f-4f06-ba3d-fa5b04ba618f admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:52:25.249487 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb4ae56-c61f-4f06-ba3d-fa5b04ba618f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:52:25.251402 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb4ae56-c61f-4f06-ba3d-fa5b04ba618f admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:52:25.262014 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1214/2448] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 14:52:25.265933 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742942b9-ed95-4fb9-a182-8340f03a9ca7 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:52:25.276718 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742942b9-ed95-4fb9-a182-8340f03a9ca7 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:52:25.282206 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febf6830-9f02-4807-b2e9-2f21296bc889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:52:25.282838 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf6830-9f02-4807-b2e9-2f21296bc889 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:52:25.283149 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf6830-9f02-4807-b2e9-2f21296bc889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:52:25.283452 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf6830-9f02-4807-b2e9-2f21296bc889 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:52:25.284121 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febf6830-9f02-4807-b2e9-2f21296bc889 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:52:25.285781 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febf6830-9f02-4807-b2e9-2f21296bc889 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:52:25.297890 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1235/2449] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 14:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 14:52:25.301692 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb4ae56-c61f-4f06-ba3d-fa5b04ba618f admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:52:25.312390 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb4ae56-c61f-4f06-ba3d-fa5b04ba618f admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:52:25.317387 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdcab50-965c-4ba7-a501-53a25fabd582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:52:25.318036 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdcab50-965c-4ba7-a501-53a25fabd582 admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:52:25.318318 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdcab50-965c-4ba7-a501-53a25fabd582 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:52:25.318595 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdcab50-965c-4ba7-a501-53a25fabd582 admin admin] PATH_INFO: `/v3/projects/8f70b782c3a8497083a6ea69a8128a0f/users/533681bb9ea140da88851b6e54ed7dad/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:52:25.333880 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdcab50-965c-4ba7-a501-53a25fabd582 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f70b782c3a8497083a6ea69a8128a0f, user_id=533681bb9ea140da88851b6e54ed7dad, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:52:25.337356 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdcab50-965c-4ba7-a501-53a25fabd582 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:52:25.349622 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1215/2450] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 14:52:25 2022] PUT /identity/v3/projects/8f70b782c3a8497083a6ea69a8128a0f/users/533681bb9ea140da88851b6e54ed7dad/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:52:25.359948 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d25d7c-fb19-4f96-9737-4cda3ac1d5d3 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:52:25.360339 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d25d7c-fb19-4f96-9737-4cda3ac1d5d3 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:52:25.360681 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d25d7c-fb19-4f96-9737-4cda3ac1d5d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:52:25.389205 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-69d25d7c-fb19-4f96-9737-4cda3ac1d5d3 None None] MFA Rules not processed for user `533681bb9ea140da88851b6e54ed7dad`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:52:25.390422 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-69d25d7c-fb19-4f96-9737-4cda3ac1d5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:52:25.417597 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:52:25.458013 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1236/2451] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:52:25 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:52:25.468881 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdcab50-965c-4ba7-a501-53a25fabd582 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:52:25.470621 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-efdcab50-965c-4ba7-a501-53a25fabd582 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:52:25.543617 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdcab50-965c-4ba7-a501-53a25fabd582 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:52:25.547224 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46584942-dc3f-44e5-a3e7-9df20d6b5e02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:52:25.547825 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46584942-dc3f-44e5-a3e7-9df20d6b5e02 service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:52:25.548128 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46584942-dc3f-44e5-a3e7-9df20d6b5e02 service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:52:25.548396 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46584942-dc3f-44e5-a3e7-9df20d6b5e02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:52:25.557279 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46584942-dc3f-44e5-a3e7-9df20d6b5e02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:52:25.558721 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46584942-dc3f-44e5-a3e7-9df20d6b5e02 service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:52:25.567544 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1216/2452] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:52:25 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:52:31.050164 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d25d7c-fb19-4f96-9737-4cda3ac1d5d3 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:52:31.051617 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-69d25d7c-fb19-4f96-9737-4cda3ac1d5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:52:31.139289 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d25d7c-fb19-4f96-9737-4cda3ac1d5d3 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:52:31.144819 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4efcb4-f5a3-468a-affa-70e799dc4d20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:52:31.145424 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4efcb4-f5a3-468a-affa-70e799dc4d20 service nova] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:52:31.145706 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4efcb4-f5a3-468a-affa-70e799dc4d20 service nova] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:52:31.145974 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4efcb4-f5a3-468a-affa-70e799dc4d20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:52:31.147766 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-8c4efcb4-f5a3-468a-affa-70e799dc4d20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:52:31.244827 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4efcb4-f5a3-468a-affa-70e799dc4d20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:52:31.246431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4efcb4-f5a3-468a-affa-70e799dc4d20 service nova] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:52:31.264876 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1237/2453] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:52:31 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:52:32.209468 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46584942-dc3f-44e5-a3e7-9df20d6b5e02 service cinder] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:52:32.217785 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46584942-dc3f-44e5-a3e7-9df20d6b5e02 service cinder] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:52:32.221472 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1872cd8-a576-4fad-932e-d7f0a5d15fd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:52:32.222067 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1872cd8-a576-4fad-932e-d7f0a5d15fd9 service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:52:32.222345 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1872cd8-a576-4fad-932e-d7f0a5d15fd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:52:32.222618 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1872cd8-a576-4fad-932e-d7f0a5d15fd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:52:32.224350 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-e1872cd8-a576-4fad-932e-d7f0a5d15fd9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:52:32.304311 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1872cd8-a576-4fad-932e-d7f0a5d15fd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:52:32.305743 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1872cd8-a576-4fad-932e-d7f0a5d15fd9 service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:52:32.324886 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1217/2454] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:52:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:52:35.266245 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4efcb4-f5a3-468a-affa-70e799dc4d20 service nova] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:52:35.275943 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4efcb4-f5a3-468a-affa-70e799dc4d20 service nova] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:52:35.279576 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7145f24-73ad-49ac-a848-79f7ca31228e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:52:35.280198 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7145f24-73ad-49ac-a848-79f7ca31228e service cinder] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:52:35.280476 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7145f24-73ad-49ac-a848-79f7ca31228e service cinder] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:52:35.280747 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7145f24-73ad-49ac-a848-79f7ca31228e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:52:35.282499 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-f7145f24-73ad-49ac-a848-79f7ca31228e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:52:35.305383 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:52:35.363353 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7145f24-73ad-49ac-a848-79f7ca31228e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:52:35.364815 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7145f24-73ad-49ac-a848-79f7ca31228e service cinder] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:52:35.383875 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1238/2455] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:52:35 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:52:42.322238 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=96,keystone:INSERT=2 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:52:45.380717 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=30 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:53:30.328042 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1872cd8-a576-4fad-932e-d7f0a5d15fd9 service cinder] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:53:30.329571 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-e1872cd8-a576-4fad-932e-d7f0a5d15fd9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:30.387807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1872cd8-a576-4fad-932e-d7f0a5d15fd9 service cinder] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:53:30.391478 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73cd694-61b4-4ae9-9527-36d69eb22f68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:53:30.392076 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73cd694-61b4-4ae9-9527-36d69eb22f68 service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:53:30.392354 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73cd694-61b4-4ae9-9527-36d69eb22f68 service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:53:30.392627 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73cd694-61b4-4ae9-9527-36d69eb22f68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:53:30.394327 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-a73cd694-61b4-4ae9-9527-36d69eb22f68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:53:30.403619 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73cd694-61b4-4ae9-9527-36d69eb22f68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:53:30.405032 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73cd694-61b4-4ae9-9527-36d69eb22f68 service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:53:30.413266 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1218/2456] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:53:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:53:40.409646 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=26 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:54:13.078655 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7145f24-73ad-49ac-a848-79f7ca31228e service cinder] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:54:13.087854 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7145f24-73ad-49ac-a848-79f7ca31228e service cinder] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:54:13.091449 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1afb8bc-35b1-41fa-b13d-ebf9ea4601d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:54:13.092607 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1afb8bc-35b1-41fa-b13d-ebf9ea4601d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:13.092972 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1afb8bc-35b1-41fa-b13d-ebf9ea4601d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:13.093245 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1afb8bc-35b1-41fa-b13d-ebf9ea4601d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:13.094955 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-a1afb8bc-35b1-41fa-b13d-ebf9ea4601d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:13.110063 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1afb8bc-35b1-41fa-b13d-ebf9ea4601d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:54:13.111771 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1afb8bc-35b1-41fa-b13d-ebf9ea4601d5 service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:54:13.123122 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1239/2457] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:54:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:54:13.544902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73cd694-61b4-4ae9-9527-36d69eb22f68 service neutron] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:54:13.563302 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73cd694-61b4-4ae9-9527-36d69eb22f68 service neutron] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:54:13.569969 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee61c09-80ce-4bad-80bd-6eb2e66d2be9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:54:13.570659 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee61c09-80ce-4bad-80bd-6eb2e66d2be9 service nova] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:54:13.571028 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee61c09-80ce-4bad-80bd-6eb2e66d2be9 service nova] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:54:13.571614 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee61c09-80ce-4bad-80bd-6eb2e66d2be9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:54:13.573816 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-aee61c09-80ce-4bad-80bd-6eb2e66d2be9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:54:13.587984 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee61c09-80ce-4bad-80bd-6eb2e66d2be9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:54:13.589670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee61c09-80ce-4bad-80bd-6eb2e66d2be9 service nova] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:54:13.618036 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1219/2458] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:54:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:54:23.118230 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=6 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:54:23.596643 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=6 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:55:12.961677 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1afb8bc-35b1-41fa-b13d-ebf9ea4601d5 service neutron] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:12.963123 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-a1afb8bc-35b1-41fa-b13d-ebf9ea4601d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:12.974367 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1afb8bc-35b1-41fa-b13d-ebf9ea4601d5 service neutron] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:12.977864 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165115a7-d7fc-404a-8aaa-e8e65361bfde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:12.978446 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165115a7-d7fc-404a-8aaa-e8e65361bfde service cinder] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:12.978718 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165115a7-d7fc-404a-8aaa-e8e65361bfde service cinder] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:12.978983 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165115a7-d7fc-404a-8aaa-e8e65361bfde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:12.980632 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-165115a7-d7fc-404a-8aaa-e8e65361bfde service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:13.060174 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165115a7-d7fc-404a-8aaa-e8e65361bfde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:13.062627 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165115a7-d7fc-404a-8aaa-e8e65361bfde service cinder] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:13.081529 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1240/2459] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:55:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:55:23.079000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=41 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:55:35.598641 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee61c09-80ce-4bad-80bd-6eb2e66d2be9 service nova] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:55:35.600108 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-aee61c09-80ce-4bad-80bd-6eb2e66d2be9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:35.668186 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee61c09-80ce-4bad-80bd-6eb2e66d2be9 service nova] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:55:35.673420 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc75894-0929-4e6d-9fd3-83044b3a30d7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcf302a170934c25b39699088dac326a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:55:35.674016 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc75894-0929-4e6d-9fd3-83044b3a30d7 service placement] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:55:35.674303 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc75894-0929-4e6d-9fd3-83044b3a30d7 service placement] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:55:35.674576 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc75894-0929-4e6d-9fd3-83044b3a30d7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:55:35.676412 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-2fc75894-0929-4e6d-9fd3-83044b3a30d7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:55:35.687844 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc75894-0929-4e6d-9fd3-83044b3a30d7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:55:35.689933 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc75894-0929-4e6d-9fd3-83044b3a30d7 service placement] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:55:35.700078 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1220/2460] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:55:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:55:45.695823 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=32 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:25.337344 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165115a7-d7fc-404a-8aaa-e8e65361bfde service cinder] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:25.347092 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165115a7-d7fc-404a-8aaa-e8e65361bfde service cinder] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:25.351435 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edad7913-d292-4c8c-9ad7-8164716c463b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:25.352037 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad7913-d292-4c8c-9ad7-8164716c463b service cinder] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:25.352434 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad7913-d292-4c8c-9ad7-8164716c463b service cinder] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:25.352715 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad7913-d292-4c8c-9ad7-8164716c463b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:25.361111 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edad7913-d292-4c8c-9ad7-8164716c463b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:25.362566 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edad7913-d292-4c8c-9ad7-8164716c463b service cinder] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:25.371370 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1241/2461] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:57:25 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:57:30.683827 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc75894-0929-4e6d-9fd3-83044b3a30d7 service placement] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:30.693130 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc75894-0929-4e6d-9fd3-83044b3a30d7 service placement] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:30.697076 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08923f04-186c-414d-9792-9eeac14fb553 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:30.697704 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08923f04-186c-414d-9792-9eeac14fb553 service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:30.697981 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08923f04-186c-414d-9792-9eeac14fb553 service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:30.698251 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08923f04-186c-414d-9792-9eeac14fb553 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:30.706488 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08923f04-186c-414d-9792-9eeac14fb553 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:30.707934 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08923f04-186c-414d-9792-9eeac14fb553 service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:30.717938 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1221/2462] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:57:30 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:57:31.820999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edad7913-d292-4c8c-9ad7-8164716c463b service cinder] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:31.829660 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edad7913-d292-4c8c-9ad7-8164716c463b service cinder] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:31.833344 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a7980-1429-4060-b9bf-aee7cc9ab1b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:31.834542 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a7980-1429-4060-b9bf-aee7cc9ab1b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:31.834925 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a7980-1429-4060-b9bf-aee7cc9ab1b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:31.835287 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a7980-1429-4060-b9bf-aee7cc9ab1b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:31.843601 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728a7980-1429-4060-b9bf-aee7cc9ab1b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:31.845017 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728a7980-1429-4060-b9bf-aee7cc9ab1b4 service cinder] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:31.854106 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1242/2463] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:57:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:57:34.332242 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08923f04-186c-414d-9792-9eeac14fb553 service cinder] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:57:34.342795 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08923f04-186c-414d-9792-9eeac14fb553 service cinder] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:57:34.346581 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e245afb5-fc4e-40ee-bf3c-f182f5911433 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:57:34.347174 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e245afb5-fc4e-40ee-bf3c-f182f5911433 service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:57:34.347482 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e245afb5-fc4e-40ee-bf3c-f182f5911433 service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:57:34.347753 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e245afb5-fc4e-40ee-bf3c-f182f5911433 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:57:34.357700 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e245afb5-fc4e-40ee-bf3c-f182f5911433 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:57:34.359115 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e245afb5-fc4e-40ee-bf3c-f182f5911433 service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:57:34.369408 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1222/2464] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:57:34 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:57:41.849702 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=12 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:57:44.364190 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=12 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:19.599892 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a7980-1429-4060-b9bf-aee7cc9ab1b4 service cinder] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:19.601434 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-728a7980-1429-4060-b9bf-aee7cc9ab1b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:19.613815 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a7980-1429-4060-b9bf-aee7cc9ab1b4 service cinder] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:19.617710 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2178298-3354-4c48-8ba6-be066cf54b08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:19.618304 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2178298-3354-4c48-8ba6-be066cf54b08 service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:19.618582 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2178298-3354-4c48-8ba6-be066cf54b08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:19.618879 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2178298-3354-4c48-8ba6-be066cf54b08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:19.620604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-c2178298-3354-4c48-8ba6-be066cf54b08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:19.635083 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2178298-3354-4c48-8ba6-be066cf54b08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:19.636609 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2178298-3354-4c48-8ba6-be066cf54b08 service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:19.648150 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1243/2465] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:58:25.663947 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e245afb5-fc4e-40ee-bf3c-f182f5911433 service cinder] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:25.737317 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e245afb5-fc4e-40ee-bf3c-f182f5911433 service cinder] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:25.742500 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512cd07d-99dd-488b-9991-0a2a9139bca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:25.743128 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512cd07d-99dd-488b-9991-0a2a9139bca4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:25.743439 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512cd07d-99dd-488b-9991-0a2a9139bca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:25.743713 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512cd07d-99dd-488b-9991-0a2a9139bca4 admin admin] PATH_INFO: `/v3/users/533681bb9ea140da88851b6e54ed7dad` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:25.744898 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512cd07d-99dd-488b-9991-0a2a9139bca4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=533681bb9ea140da88851b6e54ed7dad)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:25.746849 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512cd07d-99dd-488b-9991-0a2a9139bca4 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:25.796076 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-512cd07d-99dd-488b-9991-0a2a9139bca4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '533681bb9ea140da88851b6e54ed7dad'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:25.796742 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-512cd07d-99dd-488b-9991-0a2a9139bca4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '533681bb9ea140da88851b6e54ed7dad'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:25.808364 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-512cd07d-99dd-488b-9991-0a2a9139bca4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '533681bb9ea140da88851b6e54ed7dad'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:25.815404 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-512cd07d-99dd-488b-9991-0a2a9139bca4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '533681bb9ea140da88851b6e54ed7dad'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:25.823382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-512cd07d-99dd-488b-9991-0a2a9139bca4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '533681bb9ea140da88851b6e54ed7dad'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:25.829240 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1223/2466] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 14:58:25 2022] DELETE /identity/v3/users/533681bb9ea140da88851b6e54ed7dad => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:25.833936 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a8669-cdc6-4217-a7d8-0bf0870803ee None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:25.834948 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a8669-cdc6-4217-a7d8-0bf0870803ee None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:25.834948 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a8669-cdc6-4217-a7d8-0bf0870803ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:25.854480 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-815a8669-cdc6-4217-a7d8-0bf0870803ee None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 14:58:25.855646 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-815a8669-cdc6-4217-a7d8-0bf0870803ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:25.913940 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1244/2467] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 14:58:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 14:58:25.923734 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512cd07d-99dd-488b-9991-0a2a9139bca4 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:25.925165 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-512cd07d-99dd-488b-9991-0a2a9139bca4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:25.982501 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512cd07d-99dd-488b-9991-0a2a9139bca4 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:25.986129 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d047a8-8854-48fa-8695-b59c4891c016 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:25.986720 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d047a8-8854-48fa-8695-b59c4891c016 service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:25.986995 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d047a8-8854-48fa-8695-b59c4891c016 service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:25.987292 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d047a8-8854-48fa-8695-b59c4891c016 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:25.996234 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d047a8-8854-48fa-8695-b59c4891c016 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:25.997642 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d047a8-8854-48fa-8695-b59c4891c016 service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:26.007627 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1224/2468] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:58:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:58:26.277927 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815a8669-cdc6-4217-a7d8-0bf0870803ee None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:58:26.279530 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-815a8669-cdc6-4217-a7d8-0bf0870803ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:58:26.292242 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815a8669-cdc6-4217-a7d8-0bf0870803ee None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:58:26.298437 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd5243-1b90-40e3-ae0d-d28a7d561f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:58:26.299078 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd5243-1b90-40e3-ae0d-d28a7d561f22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:58:26.299421 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd5243-1b90-40e3-ae0d-d28a7d561f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:58:26.300419 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd5243-1b90-40e3-ae0d-d28a7d561f22 admin admin] PATH_INFO: `/v3/projects/8f70b782c3a8497083a6ea69a8128a0f` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:58:26.301920 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbd5243-1b90-40e3-ae0d-d28a7d561f22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f70b782c3a8497083a6ea69a8128a0f)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:58:26.303960 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbd5243-1b90-40e3-ae0d-d28a7d561f22 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:58:26.357347 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-1cbd5243-1b90-40e3-ae0d-d28a7d561f22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f70b782c3a8497083a6ea69a8128a0f', 'request_id': 'req-1cbd5243-1b90-40e3-ae0d-d28a7d561f22'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:26.362767 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-1cbd5243-1b90-40e3-ae0d-d28a7d561f22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f70b782c3a8497083a6ea69a8128a0f', 'request_id': 'req-1cbd5243-1b90-40e3-ae0d-d28a7d561f22'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:26.369096 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-1cbd5243-1b90-40e3-ae0d-d28a7d561f22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f70b782c3a8497083a6ea69a8128a0f', 'request_id': 'req-1cbd5243-1b90-40e3-ae0d-d28a7d561f22'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:26.382488 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-1cbd5243-1b90-40e3-ae0d-d28a7d561f22 admin admin] The token cache is being invalidate because project 8f70b782c3a8497083a6ea69a8128a0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 14:58:26.382780 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-1cbd5243-1b90-40e3-ae0d-d28a7d561f22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:26.383936 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-1cbd5243-1b90-40e3-ae0d-d28a7d561f22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 14:58:26.385460 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1245/2469] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 14:58:26 2022] DELETE /identity/v3/projects/8f70b782c3a8497083a6ea69a8128a0f => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 14:58:36.002751 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:58:36.378685 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=60,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:13.802378 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d047a8-8854-48fa-8695-b59c4891c016 service neutron] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:13.811383 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d047a8-8854-48fa-8695-b59c4891c016 service neutron] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:13.864100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce4786e-cda5-4455-858f-9ee2d1de41b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:13.864710 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4786e-cda5-4455-858f-9ee2d1de41b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:13.864993 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4786e-cda5-4455-858f-9ee2d1de41b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:13.865270 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4786e-cda5-4455-858f-9ee2d1de41b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:13.867129 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-3ce4786e-cda5-4455-858f-9ee2d1de41b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:13.939715 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce4786e-cda5-4455-858f-9ee2d1de41b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:13.941158 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce4786e-cda5-4455-858f-9ee2d1de41b7 service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:13.953166 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1225/2470] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:59:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:59:23.948323 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=54 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 14:59:38.286688 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd5243-1b90-40e3-ae0d-d28a7d561f22 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 14:59:38.288454 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-1cbd5243-1b90-40e3-ae0d-d28a7d561f22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:38.300653 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd5243-1b90-40e3-ae0d-d28a7d561f22 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 14:59:38.304994 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e938604b-4efd-42b1-8f65-86e7567d7ad9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 14:59:38.305579 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938604b-4efd-42b1-8f65-86e7567d7ad9 service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 14:59:38.305854 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938604b-4efd-42b1-8f65-86e7567d7ad9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 14:59:38.306120 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938604b-4efd-42b1-8f65-86e7567d7ad9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 14:59:38.307851 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-e938604b-4efd-42b1-8f65-86e7567d7ad9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 14:59:38.317013 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e938604b-4efd-42b1-8f65-86e7567d7ad9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 14:59:38.318420 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e938604b-4efd-42b1-8f65-86e7567d7ad9 service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 14:59:38.327286 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1246/2471] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 14:59:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 14:59:48.323817 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=6 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:00:13.411741 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce4786e-cda5-4455-858f-9ee2d1de41b7 service neutron] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:00:13.413200 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-3ce4786e-cda5-4455-858f-9ee2d1de41b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:00:13.427042 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce4786e-cda5-4455-858f-9ee2d1de41b7 service neutron] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:00:13.433191 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862091bc-ffe6-4a16-8e7e-1e2433ad7c22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:00:13.433789 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862091bc-ffe6-4a16-8e7e-1e2433ad7c22 service nova] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:00:13.434068 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862091bc-ffe6-4a16-8e7e-1e2433ad7c22 service nova] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:00:13.434340 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862091bc-ffe6-4a16-8e7e-1e2433ad7c22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:00:13.436025 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-862091bc-ffe6-4a16-8e7e-1e2433ad7c22 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:00:13.495367 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862091bc-ffe6-4a16-8e7e-1e2433ad7c22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:00:13.496867 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862091bc-ffe6-4a16-8e7e-1e2433ad7c22 service nova] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:00:13.505874 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1226/2472] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:00:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:00:23.501763 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=28 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:00:41.564043 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e938604b-4efd-42b1-8f65-86e7567d7ad9 service neutron] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:00:41.565456 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-e938604b-4efd-42b1-8f65-86e7567d7ad9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:00:41.701856 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e938604b-4efd-42b1-8f65-86e7567d7ad9 service neutron] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:00:41.707199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca76cd-1d55-42f9-808a-866405e1df4d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcf302a170934c25b39699088dac326a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:00:41.707835 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca76cd-1d55-42f9-808a-866405e1df4d service placement] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:00:41.708119 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca76cd-1d55-42f9-808a-866405e1df4d service placement] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:00:41.708394 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca76cd-1d55-42f9-808a-866405e1df4d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:00:41.710169 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-66ca76cd-1d55-42f9-808a-866405e1df4d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:00:41.722783 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ca76cd-1d55-42f9-808a-866405e1df4d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:00:41.724708 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ca76cd-1d55-42f9-808a-866405e1df4d service placement] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:00:41.736795 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1247/2473] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:00:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:00:51.731686 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=58 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:02:30.756958 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862091bc-ffe6-4a16-8e7e-1e2433ad7c22 service nova] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:30.758400 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-862091bc-ffe6-4a16-8e7e-1e2433ad7c22 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:30.828826 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862091bc-ffe6-4a16-8e7e-1e2433ad7c22 service nova] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:30.835232 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f17ab-7022-4931-a613-d8811f9b0e7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:30.835841 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f17ab-7022-4931-a613-d8811f9b0e7e service nova] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:30.836121 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f17ab-7022-4931-a613-d8811f9b0e7e service nova] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:30.836392 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f17ab-7022-4931-a613-d8811f9b0e7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:30.838084 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-650f17ab-7022-4931-a613-d8811f9b0e7e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:30.894640 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650f17ab-7022-4931-a613-d8811f9b0e7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:30.897255 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650f17ab-7022-4931-a613-d8811f9b0e7e service nova] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:30.906169 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1227/2474] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:02:30 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:02:34.581033 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca76cd-1d55-42f9-808a-866405e1df4d service placement] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:34.582555 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-66ca76cd-1d55-42f9-808a-866405e1df4d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:34.671063 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca76cd-1d55-42f9-808a-866405e1df4d service placement] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:34.674722 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b801ce25-4558-40b0-abaa-922e1610d51c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:34.675361 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b801ce25-4558-40b0-abaa-922e1610d51c service cinder] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:34.675643 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b801ce25-4558-40b0-abaa-922e1610d51c service cinder] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:34.675918 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b801ce25-4558-40b0-abaa-922e1610d51c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:34.678303 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-b801ce25-4558-40b0-abaa-922e1610d51c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:34.758842 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b801ce25-4558-40b0-abaa-922e1610d51c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:34.760212 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b801ce25-4558-40b0-abaa-922e1610d51c service cinder] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:34.778353 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1248/2475] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:02:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:02:34.856234 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f17ab-7022-4931-a613-d8811f9b0e7e service nova] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:34.865727 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b801ce25-4558-40b0-abaa-922e1610d51c service cinder] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:02:34.866529 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f17ab-7022-4931-a613-d8811f9b0e7e service nova] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:34.870154 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7c56fd-53bd-432c-8896-9406c37e05da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:34.870875 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c56fd-53bd-432c-8896-9406c37e05da service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:34.871156 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c56fd-53bd-432c-8896-9406c37e05da service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:34.871459 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c56fd-53bd-432c-8896-9406c37e05da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:34.873375 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-7b7c56fd-53bd-432c-8896-9406c37e05da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:34.878236 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b801ce25-4558-40b0-abaa-922e1610d51c service cinder] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:02:34.883938 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f96362-e1ee-457d-bf14-84e237011114 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:02:34.884535 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f96362-e1ee-457d-bf14-84e237011114 service nova] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:02:34.884813 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f96362-e1ee-457d-bf14-84e237011114 service nova] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:02:34.885086 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f96362-e1ee-457d-bf14-84e237011114 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:02:34.886806 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-d5f96362-e1ee-457d-bf14-84e237011114 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:02:34.933276 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7c56fd-53bd-432c-8896-9406c37e05da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:34.934719 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7c56fd-53bd-432c-8896-9406c37e05da service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:34.943822 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1228/2476] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:02:34 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:02:34.944082 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:02:34.949208 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f96362-e1ee-457d-bf14-84e237011114 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:02:34.950802 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f96362-e1ee-457d-bf14-84e237011114 service nova] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:02:34.961075 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1249/2477] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:02:34 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:02:44.939559 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=81 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:02:44.955790 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=3 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:03:20.352257 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7c56fd-53bd-432c-8896-9406c37e05da service cinder] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:03:20.353664 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-7b7c56fd-53bd-432c-8896-9406c37e05da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:03:20.387705 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7c56fd-53bd-432c-8896-9406c37e05da service cinder] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:03:20.391461 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a69df3-d86b-4049-ac1c-a2de3411cac8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:03:20.392065 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a69df3-d86b-4049-ac1c-a2de3411cac8 service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:03:20.392347 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a69df3-d86b-4049-ac1c-a2de3411cac8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:03:20.392619 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a69df3-d86b-4049-ac1c-a2de3411cac8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:03:20.394356 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-32a69df3-d86b-4049-ac1c-a2de3411cac8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:03:20.422793 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a69df3-d86b-4049-ac1c-a2de3411cac8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:03:20.424270 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a69df3-d86b-4049-ac1c-a2de3411cac8 service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:03:20.435649 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1229/2478] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:03:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:03:30.430351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=20 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:04:14.493197 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f96362-e1ee-457d-bf14-84e237011114 service nova] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:14.502497 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f96362-e1ee-457d-bf14-84e237011114 service nova] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:14.506991 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ae5f6-6a86-410b-8896-cf3e4bab7aad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:14.507625 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ae5f6-6a86-410b-8896-cf3e4bab7aad service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:14.507906 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ae5f6-6a86-410b-8896-cf3e4bab7aad service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:14.508211 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ae5f6-6a86-410b-8896-cf3e4bab7aad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:14.518951 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72ae5f6-6a86-410b-8896-cf3e4bab7aad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:14.520405 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72ae5f6-6a86-410b-8896-cf3e4bab7aad service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:14.531680 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1250/2479] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:04:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:04:24.526897 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=6 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:04:35.157246 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2957fb9c-9ff4-4a00-baf4-969e913d839b None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:35.157854 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2957fb9c-9ff4-4a00-baf4-969e913d839b None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:35.157854 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2957fb9c-9ff4-4a00-baf4-969e913d839b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:35.217604 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-2957fb9c-9ff4-4a00-baf4-969e913d839b None None] MFA Rules not processed for user `6270e19d6245453ab79555a79532faba`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:04:35.219619 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-2957fb9c-9ff4-4a00-baf4-969e913d839b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:35.239085 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1230/2480] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:04:35 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:04:35.782535 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ae5f6-6a86-410b-8896-cf3e4bab7aad service neutron] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:35.791771 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ae5f6-6a86-410b-8896-cf3e4bab7aad service neutron] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:35.795507 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d90347-4001-4ddb-a8e4-34dc463c450f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:35.796098 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d90347-4001-4ddb-a8e4-34dc463c450f service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:35.796391 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d90347-4001-4ddb-a8e4-34dc463c450f service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:35.796645 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d90347-4001-4ddb-a8e4-34dc463c450f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:35.798332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-83d90347-4001-4ddb-a8e4-34dc463c450f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:35.808406 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d90347-4001-4ddb-a8e4-34dc463c450f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:35.809822 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d90347-4001-4ddb-a8e4-34dc463c450f service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:35.817695 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1251/2481] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:04:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:04:36.142641 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2957fb9c-9ff4-4a00-baf4-969e913d839b None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:36.156164 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2957fb9c-9ff4-4a00-baf4-969e913d839b None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:36.161105 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375d4063-1633-451a-bd77-b9b55513535a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcf302a170934c25b39699088dac326a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:36.161703 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d4063-1633-451a-bd77-b9b55513535a service placement] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:36.161975 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d4063-1633-451a-bd77-b9b55513535a service placement] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:36.162243 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d4063-1633-451a-bd77-b9b55513535a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:36.165304 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-375d4063-1633-451a-bd77-b9b55513535a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:36.178201 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375d4063-1633-451a-bd77-b9b55513535a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:36.179904 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375d4063-1633-451a-bd77-b9b55513535a service placement] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:36.190285 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1231/2482] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:04:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:04:38.245625 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d90347-4001-4ddb-a8e4-34dc463c450f service neutron] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:38.254814 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d90347-4001-4ddb-a8e4-34dc463c450f service neutron] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:38.259080 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16630d72-fbe0-438c-9240-6ee6f75ad8b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:38.259728 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16630d72-fbe0-438c-9240-6ee6f75ad8b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:38.260001 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16630d72-fbe0-438c-9240-6ee6f75ad8b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:38.260473 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16630d72-fbe0-438c-9240-6ee6f75ad8b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:38.279024 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16630d72-fbe0-438c-9240-6ee6f75ad8b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:38.280570 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16630d72-fbe0-438c-9240-6ee6f75ad8b1 service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:38.294301 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1252/2483] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:04:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:04:38.868478 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375d4063-1633-451a-bd77-b9b55513535a service placement] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:38.881717 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375d4063-1633-451a-bd77-b9b55513535a service placement] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:38.888092 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c95e9-5aa3-435c-8309-a1a8a5b05e9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:38.888695 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c95e9-5aa3-435c-8309-a1a8a5b05e9d service nova] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:38.888978 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c95e9-5aa3-435c-8309-a1a8a5b05e9d service nova] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:38.889252 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c95e9-5aa3-435c-8309-a1a8a5b05e9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:38.891001 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-7c6c95e9-5aa3-435c-8309-a1a8a5b05e9d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:38.904633 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6c95e9-5aa3-435c-8309-a1a8a5b05e9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:38.906232 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6c95e9-5aa3-435c-8309-a1a8a5b05e9d service nova] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:38.917598 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1232/2484] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:04:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:04:44.114734 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16630d72-fbe0-438c-9240-6ee6f75ad8b1 service neutron] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:04:44.128016 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16630d72-fbe0-438c-9240-6ee6f75ad8b1 service neutron] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:04:44.134188 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b0f02-f7a7-4c74-a81c-5ce4e9c06ea6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:04:44.134788 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b0f02-f7a7-4c74-a81c-5ce4e9c06ea6 service nova] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:04:44.135061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b0f02-f7a7-4c74-a81c-5ce4e9c06ea6 service nova] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:04:44.135374 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b0f02-f7a7-4c74-a81c-5ce4e9c06ea6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:04:44.137513 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-0d0b0f02-f7a7-4c74-a81c-5ce4e9c06ea6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:04:44.150193 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0b0f02-f7a7-4c74-a81c-5ce4e9c06ea6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:04:44.151822 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0b0f02-f7a7-4c74-a81c-5ce4e9c06ea6 service nova] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:04:44.162401 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1253/2485] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:04:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:04:48.913863 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=31 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:04:54.157981 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=18 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:05:17.511158 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c95e9-5aa3-435c-8309-a1a8a5b05e9d service nova] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:05:17.512578 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-7c6c95e9-5aa3-435c-8309-a1a8a5b05e9d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:05:17.523985 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c95e9-5aa3-435c-8309-a1a8a5b05e9d service nova] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:05:17.528144 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b379df87-4045-43aa-9bed-847439ea74ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:05:17.528735 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b379df87-4045-43aa-9bed-847439ea74ec service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:05:17.529015 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b379df87-4045-43aa-9bed-847439ea74ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:05:17.529289 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b379df87-4045-43aa-9bed-847439ea74ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:05:17.537495 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b379df87-4045-43aa-9bed-847439ea74ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:05:17.538925 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b379df87-4045-43aa-9bed-847439ea74ec service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:05:17.565937 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1233/2486] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:05:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:05:27.563423 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=13 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:05:47.237099 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b0f02-f7a7-4c74-a81c-5ce4e9c06ea6 service nova] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:05:47.238812 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-0d0b0f02-f7a7-4c74-a81c-5ce4e9c06ea6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:05:47.253316 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b0f02-f7a7-4c74-a81c-5ce4e9c06ea6 service nova] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:05:47.258385 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e003ddc4-2dc7-4fa8-b496-baeab9a4f14f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcf302a170934c25b39699088dac326a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:05:47.259015 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e003ddc4-2dc7-4fa8-b496-baeab9a4f14f service placement] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:05:47.259389 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e003ddc4-2dc7-4fa8-b496-baeab9a4f14f service placement] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:05:47.259667 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e003ddc4-2dc7-4fa8-b496-baeab9a4f14f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:05:47.269304 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e003ddc4-2dc7-4fa8-b496-baeab9a4f14f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:05:47.270885 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e003ddc4-2dc7-4fa8-b496-baeab9a4f14f service placement] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:05:47.282100 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1254/2487] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:05:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:05:57.276788 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=6 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:07:17.891504 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b379df87-4045-43aa-9bed-847439ea74ec service cinder] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:07:17.901184 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b379df87-4045-43aa-9bed-847439ea74ec service cinder] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:07:17.904801 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c790b167-3eb8-4eff-b86c-50ee192c61ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:07:17.905390 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c790b167-3eb8-4eff-b86c-50ee192c61ab service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:07:17.905668 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c790b167-3eb8-4eff-b86c-50ee192c61ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:07:17.905937 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c790b167-3eb8-4eff-b86c-50ee192c61ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:07:17.908775 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-c790b167-3eb8-4eff-b86c-50ee192c61ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:07:17.994006 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c790b167-3eb8-4eff-b86c-50ee192c61ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:07:17.995459 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c790b167-3eb8-4eff-b86c-50ee192c61ab service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:07:18.014017 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1234/2488] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:07:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:07:21.630504 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e003ddc4-2dc7-4fa8-b496-baeab9a4f14f service placement] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:07:21.641835 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e003ddc4-2dc7-4fa8-b496-baeab9a4f14f service placement] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:07:21.647329 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d1df0a-56f7-47bf-8fbd-2251d61ce91b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcf302a170934c25b39699088dac326a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:07:21.647941 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1df0a-56f7-47bf-8fbd-2251d61ce91b service placement] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:07:21.648416 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1df0a-56f7-47bf-8fbd-2251d61ce91b service placement] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:07:21.648881 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1df0a-56f7-47bf-8fbd-2251d61ce91b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:07:21.651406 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-f3d1df0a-56f7-47bf-8fbd-2251d61ce91b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:07:21.666737 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d1df0a-56f7-47bf-8fbd-2251d61ce91b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:07:21.668624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d1df0a-56f7-47bf-8fbd-2251d61ce91b service placement] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:07:21.680680 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1255/2489] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:07:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:07:21.686510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c790b167-3eb8-4eff-b86c-50ee192c61ab service neutron] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:07:21.688117 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-c790b167-3eb8-4eff-b86c-50ee192c61ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:07:21.703574 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c790b167-3eb8-4eff-b86c-50ee192c61ab service neutron] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:07:21.708652 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98db9180-070e-4665-b7e8-7ef5e99e9d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:07:21.709284 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db9180-070e-4665-b7e8-7ef5e99e9d38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:07:21.709569 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db9180-070e-4665-b7e8-7ef5e99e9d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:07:21.709846 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db9180-070e-4665-b7e8-7ef5e99e9d38 admin admin] PATH_INFO: `/v3/users/ca9bafc5ff4e43719f37e62cad946cf7` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:07:21.710988 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98db9180-070e-4665-b7e8-7ef5e99e9d38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca9bafc5ff4e43719f37e62cad946cf7)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:07:21.712932 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98db9180-070e-4665-b7e8-7ef5e99e9d38 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:07:21.759902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-98db9180-070e-4665-b7e8-7ef5e99e9d38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca9bafc5ff4e43719f37e62cad946cf7'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:07:21.760563 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-98db9180-070e-4665-b7e8-7ef5e99e9d38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca9bafc5ff4e43719f37e62cad946cf7'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:07:21.772805 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-98db9180-070e-4665-b7e8-7ef5e99e9d38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca9bafc5ff4e43719f37e62cad946cf7'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:07:21.779659 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-98db9180-070e-4665-b7e8-7ef5e99e9d38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca9bafc5ff4e43719f37e62cad946cf7'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:07:21.789938 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-98db9180-070e-4665-b7e8-7ef5e99e9d38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca9bafc5ff4e43719f37e62cad946cf7'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:07:21.795868 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1235/2490] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 15:07:21 2022] DELETE /identity/v3/users/ca9bafc5ff4e43719f37e62cad946cf7 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:07:21.967991 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d1df0a-56f7-47bf-8fbd-2251d61ce91b service placement] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:07:21.979740 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d1df0a-56f7-47bf-8fbd-2251d61ce91b service placement] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:07:22.037261 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b4f80d-230e-4d2b-84ce-7f5be72bfa6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:07:22.037890 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b4f80d-230e-4d2b-84ce-7f5be72bfa6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:07:22.038172 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b4f80d-230e-4d2b-84ce-7f5be72bfa6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:07:22.038445 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b4f80d-230e-4d2b-84ce-7f5be72bfa6d admin admin] PATH_INFO: `/v3/projects/04efb141e084405992bfc0418a74da2a` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:07:22.039673 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b4f80d-230e-4d2b-84ce-7f5be72bfa6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=04efb141e084405992bfc0418a74da2a)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:07:22.041608 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b4f80d-230e-4d2b-84ce-7f5be72bfa6d admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:07:22.087138 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-41b4f80d-230e-4d2b-84ce-7f5be72bfa6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04efb141e084405992bfc0418a74da2a', 'request_id': 'req-41b4f80d-230e-4d2b-84ce-7f5be72bfa6d'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:07:22.092535 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-41b4f80d-230e-4d2b-84ce-7f5be72bfa6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04efb141e084405992bfc0418a74da2a', 'request_id': 'req-41b4f80d-230e-4d2b-84ce-7f5be72bfa6d'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:07:22.099266 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-41b4f80d-230e-4d2b-84ce-7f5be72bfa6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04efb141e084405992bfc0418a74da2a', 'request_id': 'req-41b4f80d-230e-4d2b-84ce-7f5be72bfa6d'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:07:22.111871 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-41b4f80d-230e-4d2b-84ce-7f5be72bfa6d admin admin] The token cache is being invalidate because project 04efb141e084405992bfc0418a74da2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:07:22.112191 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-41b4f80d-230e-4d2b-84ce-7f5be72bfa6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:07:22.112808 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-41b4f80d-230e-4d2b-84ce-7f5be72bfa6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:07:22.114273 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1256/2491] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 15:07:21 2022] DELETE /identity/v3/projects/04efb141e084405992bfc0418a74da2a => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:07:22.120636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b1938-f9cc-4395-9967-a9e696ba68d0 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:07:22.120924 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b1938-f9cc-4395-9967-a9e696ba68d0 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:07:22.121198 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b1938-f9cc-4395-9967-a9e696ba68d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:07:22.180441 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:07:22.183592 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-dd1b1938-f9cc-4395-9967-a9e696ba68d0 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:07:22.185366 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-dd1b1938-f9cc-4395-9967-a9e696ba68d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:07:22.245843 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1236/2492] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:07:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:07:22.254556 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2174c12-d63b-465c-958f-bdf0957ee74a None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:07:22.254841 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2174c12-d63b-465c-958f-bdf0957ee74a None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:07:22.255112 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2174c12-d63b-465c-958f-bdf0957ee74a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:07:22.275297 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-a2174c12-d63b-465c-958f-bdf0957ee74a None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:07:22.276488 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-a2174c12-d63b-465c-958f-bdf0957ee74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:07:22.284620 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1257/2493] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:07:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:07:22.288751 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b1938-f9cc-4395-9967-a9e696ba68d0 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:07:22.298429 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b1938-f9cc-4395-9967-a9e696ba68d0 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:07:22.303317 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20263dd5-0e9d-49b4-94ec-09b36526cf6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:07:22.303970 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20263dd5-0e9d-49b4-94ec-09b36526cf6a admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:07:22.304253 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20263dd5-0e9d-49b4-94ec-09b36526cf6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:07:22.304530 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20263dd5-0e9d-49b4-94ec-09b36526cf6a admin admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:07:22.305181 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20263dd5-0e9d-49b4-94ec-09b36526cf6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:07:22.305487 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20263dd5-0e9d-49b4-94ec-09b36526cf6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:07:22.307129 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20263dd5-0e9d-49b4-94ec-09b36526cf6a admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:07:22.319140 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1237/2494] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 15:07:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:07:22.323687 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2174c12-d63b-465c-958f-bdf0957ee74a None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:07:22.333727 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2174c12-d63b-465c-958f-bdf0957ee74a None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:07:22.338435 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2681580e-2162-4d32-8599-eb14de9c8025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:07:22.339018 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681580e-2162-4d32-8599-eb14de9c8025 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:07:22.339329 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681580e-2162-4d32-8599-eb14de9c8025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:07:22.339599 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681580e-2162-4d32-8599-eb14de9c8025 admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:07:22.340221 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2681580e-2162-4d32-8599-eb14de9c8025 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:07:22.342116 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2681580e-2162-4d32-8599-eb14de9c8025 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:07:22.379861 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1258/2495] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 15:07:22 2022] POST /identity/v3/projects => generated 381 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:07:22.383954 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20263dd5-0e9d-49b4-94ec-09b36526cf6a admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:07:22.394546 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20263dd5-0e9d-49b4-94ec-09b36526cf6a admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:07:22.449890 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a300bcd6-17ac-4508-ae8c-36d67eb478e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:07:22.450491 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a300bcd6-17ac-4508-ae8c-36d67eb478e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:07:22.450771 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a300bcd6-17ac-4508-ae8c-36d67eb478e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:07:22.451049 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a300bcd6-17ac-4508-ae8c-36d67eb478e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:07:22.451744 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a300bcd6-17ac-4508-ae8c-36d67eb478e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:07:22.454321 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a300bcd6-17ac-4508-ae8c-36d67eb478e2 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:07:22.475903 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1238/2496] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 15:07:22 2022] POST /identity/v3/users => generated 291 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:07:22.479918 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2681580e-2162-4d32-8599-eb14de9c8025 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:07:22.490188 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2681580e-2162-4d32-8599-eb14de9c8025 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:07:22.494785 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f19ac6a-0baf-45bb-be07-5c872edbe1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:07:22.495408 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f19ac6a-0baf-45bb-be07-5c872edbe1b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:07:22.495687 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f19ac6a-0baf-45bb-be07-5c872edbe1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:07:22.495964 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f19ac6a-0baf-45bb-be07-5c872edbe1b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:07:22.496596 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f19ac6a-0baf-45bb-be07-5c872edbe1b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:07:22.498162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f19ac6a-0baf-45bb-be07-5c872edbe1b3 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:07:22.508524 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1259/2497] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:07:22.512342 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a300bcd6-17ac-4508-ae8c-36d67eb478e2 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:07:22.523032 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a300bcd6-17ac-4508-ae8c-36d67eb478e2 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:07:22.527729 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6528df-c1b1-4942-a0bd-cc3a616b96e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:07:22.528323 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6528df-c1b1-4942-a0bd-cc3a616b96e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:07:22.528605 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6528df-c1b1-4942-a0bd-cc3a616b96e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:07:22.528880 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6528df-c1b1-4942-a0bd-cc3a616b96e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:07:22.529522 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6528df-c1b1-4942-a0bd-cc3a616b96e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:07:22.531153 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6528df-c1b1-4942-a0bd-cc3a616b96e5 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:07:22.542258 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1239/2498] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:07:22.546087 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f19ac6a-0baf-45bb-be07-5c872edbe1b3 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:07:22.556921 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f19ac6a-0baf-45bb-be07-5c872edbe1b3 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:07:22.561649 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72656987-cb03-43f2-8a24-1216d4000a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:07:22.562997 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72656987-cb03-43f2-8a24-1216d4000a18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:07:22.563310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72656987-cb03-43f2-8a24-1216d4000a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:07:22.563587 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72656987-cb03-43f2-8a24-1216d4000a18 admin admin] PATH_INFO: `/v3/projects/1827feb276a945bb86a2f853126dfb20/users/ecec9a5a18b24ae7ab055c37ea85ca18/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:07:22.578952 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72656987-cb03-43f2-8a24-1216d4000a18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1827feb276a945bb86a2f853126dfb20, user_id=ecec9a5a18b24ae7ab055c37ea85ca18, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:07:22.581636 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72656987-cb03-43f2-8a24-1216d4000a18 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:07:22.594421 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1260/2499] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 15:07:22 2022] PUT /identity/v3/projects/1827feb276a945bb86a2f853126dfb20/users/ecec9a5a18b24ae7ab055c37ea85ca18/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:07:22.599670 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be78fff6-ae11-4e7b-8cac-3db54fdaae9f None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:07:22.600050 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be78fff6-ae11-4e7b-8cac-3db54fdaae9f None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:07:22.600389 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be78fff6-ae11-4e7b-8cac-3db54fdaae9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:07:22.619960 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-be78fff6-ae11-4e7b-8cac-3db54fdaae9f None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:07:22.621157 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-be78fff6-ae11-4e7b-8cac-3db54fdaae9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:07:22.679773 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1240/2500] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:07:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:07:22.690500 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72656987-cb03-43f2-8a24-1216d4000a18 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:07:22.692429 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-72656987-cb03-43f2-8a24-1216d4000a18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:07:22.699299 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:07:22.757596 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72656987-cb03-43f2-8a24-1216d4000a18 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:07:22.761149 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a098e1-116c-4e74-bb9f-84a76247edae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:07:22.761747 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a098e1-116c-4e74-bb9f-84a76247edae service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:07:22.762021 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a098e1-116c-4e74-bb9f-84a76247edae service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:07:22.762288 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a098e1-116c-4e74-bb9f-84a76247edae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:07:22.771972 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a098e1-116c-4e74-bb9f-84a76247edae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:07:22.773377 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a098e1-116c-4e74-bb9f-84a76247edae service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:07:22.783685 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1261/2501] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:07:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:07:26.915053 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d4a85-4c6d-421b-b3c6-7d2e7a05ba6f None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:07:26.915603 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d4a85-4c6d-421b-b3c6-7d2e7a05ba6f None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:07:26.915729 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d4a85-4c6d-421b-b3c6-7d2e7a05ba6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:07:26.947021 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:07:26.950946 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-a41d4a85-4c6d-421b-b3c6-7d2e7a05ba6f None None] MFA Rules not processed for user `ecec9a5a18b24ae7ab055c37ea85ca18`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:07:26.953707 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-a41d4a85-4c6d-421b-b3c6-7d2e7a05ba6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:07:27.027785 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1241/2502] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:07:26 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:07:27.042018 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a098e1-116c-4e74-bb9f-84a76247edae service neutron] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:07:27.043609 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-59a098e1-116c-4e74-bb9f-84a76247edae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:07:27.121516 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a098e1-116c-4e74-bb9f-84a76247edae service neutron] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:07:27.128003 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a612bd7b-dd2a-4dca-84b6-08b4f67d9506 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:07:27.128628 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a612bd7b-dd2a-4dca-84b6-08b4f67d9506 service nova] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:07:27.128910 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a612bd7b-dd2a-4dca-84b6-08b4f67d9506 service nova] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:07:27.129185 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a612bd7b-dd2a-4dca-84b6-08b4f67d9506 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:07:27.137778 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a612bd7b-dd2a-4dca-84b6-08b4f67d9506 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:07:27.139436 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a612bd7b-dd2a-4dca-84b6-08b4f67d9506 service nova] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:07:27.149091 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1262/2503] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:07:27 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:07:28.492591 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e131c656-5092-4ac5-b7b2-173ffc5a1cd6 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:07:28.493106 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e131c656-5092-4ac5-b7b2-173ffc5a1cd6 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:07:28.493248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e131c656-5092-4ac5-b7b2-173ffc5a1cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:07:28.513211 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-e131c656-5092-4ac5-b7b2-173ffc5a1cd6 None None] MFA Rules not processed for user `ecec9a5a18b24ae7ab055c37ea85ca18`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:07:28.514431 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-e131c656-5092-4ac5-b7b2-173ffc5a1cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:07:28.523847 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1242/2504] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:07:28 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:07:37.144750 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=58 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:07:38.507378 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=29 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:08:35.698960 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a612bd7b-dd2a-4dca-84b6-08b4f67d9506 service nova] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:08:35.700393 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-a612bd7b-dd2a-4dca-84b6-08b4f67d9506 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:08:35.761879 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a612bd7b-dd2a-4dca-84b6-08b4f67d9506 service nova] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:08:35.765622 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0e72cc-41fe-44eb-8516-cb2594fff3c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:08:35.766280 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0e72cc-41fe-44eb-8516-cb2594fff3c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:08:35.767232 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0e72cc-41fe-44eb-8516-cb2594fff3c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:08:35.767554 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0e72cc-41fe-44eb-8516-cb2594fff3c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:08:35.769529 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-6c0e72cc-41fe-44eb-8516-cb2594fff3c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:08:35.831722 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0e72cc-41fe-44eb-8516-cb2594fff3c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:08:35.833155 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0e72cc-41fe-44eb-8516-cb2594fff3c9 service cinder] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:08:35.842547 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1263/2505] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:08:35 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:08:35.914152 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e131c656-5092-4ac5-b7b2-173ffc5a1cd6 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:08:35.924389 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e131c656-5092-4ac5-b7b2-173ffc5a1cd6 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:08:35.928122 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c23a43-7722-4482-87a7-1b60a252add0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:08:35.928719 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c23a43-7722-4482-87a7-1b60a252add0 service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:08:35.929000 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c23a43-7722-4482-87a7-1b60a252add0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:08:35.929273 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c23a43-7722-4482-87a7-1b60a252add0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:08:35.931017 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-c8c23a43-7722-4482-87a7-1b60a252add0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:08:36.014805 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c23a43-7722-4482-87a7-1b60a252add0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:08:36.016213 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c23a43-7722-4482-87a7-1b60a252add0 service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:08:36.034169 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1243/2506] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:08:35 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:08:45.837976 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=48 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:08:46.031464 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=41 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:09:15.640032 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0e72cc-41fe-44eb-8516-cb2594fff3c9 service cinder] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:15.649470 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0e72cc-41fe-44eb-8516-cb2594fff3c9 service cinder] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:15.653015 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17232a53-f486-4a91-bba0-656b9ecdc81a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:15.653603 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17232a53-f486-4a91-bba0-656b9ecdc81a service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:15.653879 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17232a53-f486-4a91-bba0-656b9ecdc81a service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:15.654149 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17232a53-f486-4a91-bba0-656b9ecdc81a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:15.655893 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-17232a53-f486-4a91-bba0-656b9ecdc81a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:09:15.669361 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17232a53-f486-4a91-bba0-656b9ecdc81a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:15.670762 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17232a53-f486-4a91-bba0-656b9ecdc81a service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:15.682278 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1264/2507] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:09:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:09:16.856469 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c23a43-7722-4482-87a7-1b60a252add0 service cinder] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:16.857936 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-c8c23a43-7722-4482-87a7-1b60a252add0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:09:16.870253 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c23a43-7722-4482-87a7-1b60a252add0 service cinder] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:16.873810 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f34eed0-9414-475b-945d-ebbec4b4fac7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:16.874398 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34eed0-9414-475b-945d-ebbec4b4fac7 service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:16.874676 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34eed0-9414-475b-945d-ebbec4b4fac7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:16.874946 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34eed0-9414-475b-945d-ebbec4b4fac7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:16.876907 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-7f34eed0-9414-475b-945d-ebbec4b4fac7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:09:16.889979 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f34eed0-9414-475b-945d-ebbec4b4fac7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:16.891426 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f34eed0-9414-475b-945d-ebbec4b4fac7 service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:16.902374 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1244/2508] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:09:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:09:21.035671 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3e2ea-f4f0-4813-a0e8-c10c0c087962 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:21.036253 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3e2ea-f4f0-4813-a0e8-c10c0c087962 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:21.036253 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3e2ea-f4f0-4813-a0e8-c10c0c087962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:21.056805 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-7eb3e2ea-f4f0-4813-a0e8-c10c0c087962 None None] MFA Rules not processed for user `ecec9a5a18b24ae7ab055c37ea85ca18`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:09:21.058031 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-7eb3e2ea-f4f0-4813-a0e8-c10c0c087962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:09:21.064962 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1265/2509] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:09:21 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 15:09:21.372788 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f34eed0-9414-475b-945d-ebbec4b4fac7 service neutron] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:09:21.381998 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f34eed0-9414-475b-945d-ebbec4b4fac7 service neutron] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:09:21.385819 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc52550c-4c13-4e84-9e24-1651f8c54c6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:09:21.386417 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52550c-4c13-4e84-9e24-1651f8c54c6c service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:09:21.386863 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52550c-4c13-4e84-9e24-1651f8c54c6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:09:21.387053 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc52550c-4c13-4e84-9e24-1651f8c54c6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:09:21.388901 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-dc52550c-4c13-4e84-9e24-1651f8c54c6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:09:21.399523 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc52550c-4c13-4e84-9e24-1651f8c54c6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:09:21.400930 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc52550c-4c13-4e84-9e24-1651f8c54c6c service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:09:21.408843 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1245/2510] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:09:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:09:31.049645 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=9 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:09:31.405248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=12 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:10:36.108912 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb3e2ea-f4f0-4813-a0e8-c10c0c087962 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:10:36.118858 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb3e2ea-f4f0-4813-a0e8-c10c0c087962 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:10:36.122885 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783d0a2f-2b5f-48af-8edd-fd359e24713e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:10:36.123507 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d0a2f-2b5f-48af-8edd-fd359e24713e service cinder] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:10:36.123784 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d0a2f-2b5f-48af-8edd-fd359e24713e service cinder] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:10:36.124053 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d0a2f-2b5f-48af-8edd-fd359e24713e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:10:36.125783 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-783d0a2f-2b5f-48af-8edd-fd359e24713e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:10:36.195872 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783d0a2f-2b5f-48af-8edd-fd359e24713e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:10:36.197279 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783d0a2f-2b5f-48af-8edd-fd359e24713e service cinder] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:10:36.207228 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1266/2511] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:10:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:10:46.203103 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=33 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:10:48.271514 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc52550c-4c13-4e84-9e24-1651f8c54c6c service neutron] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:10:48.281015 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc52550c-4c13-4e84-9e24-1651f8c54c6c service neutron] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:10:48.284703 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a116508f-189c-4452-86be-5f00d8d5666a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:10:48.285301 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a116508f-189c-4452-86be-5f00d8d5666a service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:10:48.285580 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a116508f-189c-4452-86be-5f00d8d5666a service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:10:48.285851 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a116508f-189c-4452-86be-5f00d8d5666a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:10:48.287629 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-a116508f-189c-4452-86be-5f00d8d5666a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:10:48.306423 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a116508f-189c-4452-86be-5f00d8d5666a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:10:48.307888 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a116508f-189c-4452-86be-5f00d8d5666a service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:10:48.315896 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1246/2512] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:10:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:10:49.609272 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783d0a2f-2b5f-48af-8edd-fd359e24713e service cinder] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:10:49.610651 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-783d0a2f-2b5f-48af-8edd-fd359e24713e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:10:49.704837 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783d0a2f-2b5f-48af-8edd-fd359e24713e service cinder] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:10:49.709936 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a36b08-1f9e-4839-be7c-4657a60b212b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcf302a170934c25b39699088dac326a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:10:49.710535 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a36b08-1f9e-4839-be7c-4657a60b212b service placement] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:10:49.710813 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a36b08-1f9e-4839-be7c-4657a60b212b service placement] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:10:49.711084 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a36b08-1f9e-4839-be7c-4657a60b212b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:10:49.712837 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-99a36b08-1f9e-4839-be7c-4657a60b212b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:10:49.725114 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a36b08-1f9e-4839-be7c-4657a60b212b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:10:49.726680 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a36b08-1f9e-4839-be7c-4657a60b212b service placement] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:10:49.736616 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1267/2513] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:10:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:10:58.312339 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=10 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:10:59.732943 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=41 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:12:36.770813 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a116508f-189c-4452-86be-5f00d8d5666a service neutron] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:12:36.801156 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a116508f-189c-4452-86be-5f00d8d5666a service neutron] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:12:36.804844 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362526b4-87d2-4e84-8d36-d5c13d28854c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:12:36.806125 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362526b4-87d2-4e84-8d36-d5c13d28854c service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:12:36.806490 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362526b4-87d2-4e84-8d36-d5c13d28854c service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:12:36.806800 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362526b4-87d2-4e84-8d36-d5c13d28854c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:12:36.829440 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362526b4-87d2-4e84-8d36-d5c13d28854c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:12:36.830885 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362526b4-87d2-4e84-8d36-d5c13d28854c service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:12:36.839955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1247/2514] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:12:36 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:12:37.376202 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a36b08-1f9e-4839-be7c-4657a60b212b service placement] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:12:37.396105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a36b08-1f9e-4839-be7c-4657a60b212b service placement] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:12:37.399848 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daed52fa-319f-4ad6-8b41-6183cd5ef910 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:12:37.400459 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed52fa-319f-4ad6-8b41-6183cd5ef910 service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:12:37.400745 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed52fa-319f-4ad6-8b41-6183cd5ef910 service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:12:37.401023 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed52fa-319f-4ad6-8b41-6183cd5ef910 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:12:37.402791 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-daed52fa-319f-4ad6-8b41-6183cd5ef910 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:12:37.433292 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daed52fa-319f-4ad6-8b41-6183cd5ef910 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:12:37.434640 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daed52fa-319f-4ad6-8b41-6183cd5ef910 service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:12:37.444332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1268/2515] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:12:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:12:40.204599 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362526b4-87d2-4e84-8d36-d5c13d28854c service cinder] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:12:40.228395 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362526b4-87d2-4e84-8d36-d5c13d28854c service cinder] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:12:40.235081 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79862c1d-647c-4e54-af9d-3dc95f069f22 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcf302a170934c25b39699088dac326a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:12:40.236241 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79862c1d-647c-4e54-af9d-3dc95f069f22 service placement] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:12:40.236718 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79862c1d-647c-4e54-af9d-3dc95f069f22 service placement] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:12:40.237285 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79862c1d-647c-4e54-af9d-3dc95f069f22 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:12:40.239489 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-79862c1d-647c-4e54-af9d-3dc95f069f22 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:12:40.253262 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79862c1d-647c-4e54-af9d-3dc95f069f22 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:12:40.254907 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79862c1d-647c-4e54-af9d-3dc95f069f22 service placement] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:12:40.268449 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1248/2516] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:12:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:12:40.279575 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daed52fa-319f-4ad6-8b41-6183cd5ef910 service neutron] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:12:40.280997 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-daed52fa-319f-4ad6-8b41-6183cd5ef910 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:12:40.308918 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daed52fa-319f-4ad6-8b41-6183cd5ef910 service neutron] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:12:40.316165 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9318b4bb-bc29-474e-9955-a283303e34d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:12:40.316663 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318b4bb-bc29-474e-9955-a283303e34d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:12:40.316897 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318b4bb-bc29-474e-9955-a283303e34d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:12:40.317661 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318b4bb-bc29-474e-9955-a283303e34d2 admin admin] PATH_INFO: `/v3/users/6270e19d6245453ab79555a79532faba` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:12:40.318597 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9318b4bb-bc29-474e-9955-a283303e34d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6270e19d6245453ab79555a79532faba)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:12:40.321239 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9318b4bb-bc29-474e-9955-a283303e34d2 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:12:40.368360 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-9318b4bb-bc29-474e-9955-a283303e34d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6270e19d6245453ab79555a79532faba'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:12:40.369031 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-9318b4bb-bc29-474e-9955-a283303e34d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6270e19d6245453ab79555a79532faba'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:12:40.376067 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-9318b4bb-bc29-474e-9955-a283303e34d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6270e19d6245453ab79555a79532faba'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:12:40.382875 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-9318b4bb-bc29-474e-9955-a283303e34d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6270e19d6245453ab79555a79532faba'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:12:40.384045 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-9318b4bb-bc29-474e-9955-a283303e34d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6270e19d6245453ab79555a79532faba'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:12:40.401587 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1269/2517] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 15:12:40 2022] DELETE /identity/v3/users/6270e19d6245453ab79555a79532faba => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:12:40.569014 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79862c1d-647c-4e54-af9d-3dc95f069f22 service placement] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:12:40.581488 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79862c1d-647c-4e54-af9d-3dc95f069f22 service placement] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:12:40.639366 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac43b28-e9ae-47fc-be54-917199ad62d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:12:40.639989 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac43b28-e9ae-47fc-be54-917199ad62d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:12:40.640267 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac43b28-e9ae-47fc-be54-917199ad62d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:12:40.640536 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac43b28-e9ae-47fc-be54-917199ad62d7 admin admin] PATH_INFO: `/v3/projects/a8914c5bc86f4ee6a9797b05a6a7b12b` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:12:40.641639 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac43b28-e9ae-47fc-be54-917199ad62d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8914c5bc86f4ee6a9797b05a6a7b12b)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:12:40.643555 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac43b28-e9ae-47fc-be54-917199ad62d7 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:12:40.689415 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-4ac43b28-e9ae-47fc-be54-917199ad62d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8914c5bc86f4ee6a9797b05a6a7b12b', 'request_id': 'req-4ac43b28-e9ae-47fc-be54-917199ad62d7'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:12:40.694817 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-4ac43b28-e9ae-47fc-be54-917199ad62d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8914c5bc86f4ee6a9797b05a6a7b12b', 'request_id': 'req-4ac43b28-e9ae-47fc-be54-917199ad62d7'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:12:40.702290 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-4ac43b28-e9ae-47fc-be54-917199ad62d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8914c5bc86f4ee6a9797b05a6a7b12b', 'request_id': 'req-4ac43b28-e9ae-47fc-be54-917199ad62d7'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:12:40.714050 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-4ac43b28-e9ae-47fc-be54-917199ad62d7 admin admin] The token cache is being invalidate because project a8914c5bc86f4ee6a9797b05a6a7b12b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:12:40.714338 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-4ac43b28-e9ae-47fc-be54-917199ad62d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:12:40.714978 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-4ac43b28-e9ae-47fc-be54-917199ad62d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:12:40.716467 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1249/2518] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 15:12:40 2022] DELETE /identity/v3/projects/a8914c5bc86f4ee6a9797b05a6a7b12b => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:12:40.722164 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d013f-4eeb-45f9-a5d3-381a4a15af4d None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:12:40.722448 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d013f-4eeb-45f9-a5d3-381a4a15af4d None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:12:40.722723 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d013f-4eeb-45f9-a5d3-381a4a15af4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:12:40.742151 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-e68d013f-4eeb-45f9-a5d3-381a4a15af4d None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:12:40.744118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-e68d013f-4eeb-45f9-a5d3-381a4a15af4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:12:40.801874 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1270/2519] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:12:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 15:12:40.809991 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d353b3f8-1b5b-4200-a92e-bf7a6a0c1559 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:12:40.810306 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d353b3f8-1b5b-4200-a92e-bf7a6a0c1559 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:12:40.810578 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d353b3f8-1b5b-4200-a92e-bf7a6a0c1559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:12:40.830731 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-d353b3f8-1b5b-4200-a92e-bf7a6a0c1559 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:12:40.832028 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-d353b3f8-1b5b-4200-a92e-bf7a6a0c1559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:12:40.840395 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1250/2520] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:12:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:12:40.844484 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68d013f-4eeb-45f9-a5d3-381a4a15af4d None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:12:40.854260 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68d013f-4eeb-45f9-a5d3-381a4a15af4d None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:12:40.858810 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8693f1ef-89aa-4965-87ab-49068aec4d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:12:40.859420 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693f1ef-89aa-4965-87ab-49068aec4d7a admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:12:40.859699 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693f1ef-89aa-4965-87ab-49068aec4d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:12:40.859966 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693f1ef-89aa-4965-87ab-49068aec4d7a admin admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:12:40.860601 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8693f1ef-89aa-4965-87ab-49068aec4d7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:12:40.860902 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8693f1ef-89aa-4965-87ab-49068aec4d7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:12:40.862500 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8693f1ef-89aa-4965-87ab-49068aec4d7a admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:12:40.874310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1271/2521] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 15:12:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:12:40.878526 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d353b3f8-1b5b-4200-a92e-bf7a6a0c1559 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:12:40.888466 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d353b3f8-1b5b-4200-a92e-bf7a6a0c1559 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:12:40.893231 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd12bd29-69f9-47a9-8af4-7c7696258873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:12:40.893821 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd12bd29-69f9-47a9-8af4-7c7696258873 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:12:40.894101 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd12bd29-69f9-47a9-8af4-7c7696258873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:12:40.894374 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd12bd29-69f9-47a9-8af4-7c7696258873 admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:12:40.895014 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd12bd29-69f9-47a9-8af4-7c7696258873 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:12:40.896945 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd12bd29-69f9-47a9-8af4-7c7696258873 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:12:40.924249 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1251/2522] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 15:12:40 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:12:40.928352 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8693f1ef-89aa-4965-87ab-49068aec4d7a admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:12:40.939314 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8693f1ef-89aa-4965-87ab-49068aec4d7a admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:12:40.956492 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:12:40.998141 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5bf245-b014-474c-85d8-3eb3fcbbb76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:12:40.998747 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5bf245-b014-474c-85d8-3eb3fcbbb76c admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:12:40.999029 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5bf245-b014-474c-85d8-3eb3fcbbb76c admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:12:40.999336 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5bf245-b014-474c-85d8-3eb3fcbbb76c admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:12:40.999961 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5bf245-b014-474c-85d8-3eb3fcbbb76c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:12:41.001877 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5bf245-b014-474c-85d8-3eb3fcbbb76c admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:12:41.021527 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1272/2523] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 15:12:40 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:12:41.026012 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd12bd29-69f9-47a9-8af4-7c7696258873 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:12:41.031620 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:12:41.038016 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd12bd29-69f9-47a9-8af4-7c7696258873 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:12:41.044703 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36248c4-9058-4fe0-a1a0-328ee167c396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:12:41.045299 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36248c4-9058-4fe0-a1a0-328ee167c396 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:12:41.045578 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36248c4-9058-4fe0-a1a0-328ee167c396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:12:41.045848 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36248c4-9058-4fe0-a1a0-328ee167c396 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:12:41.046486 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36248c4-9058-4fe0-a1a0-328ee167c396 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:12:41.048132 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36248c4-9058-4fe0-a1a0-328ee167c396 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:12:41.058512 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1252/2524] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:12:41.062411 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5bf245-b014-474c-85d8-3eb3fcbbb76c admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:12:41.072469 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5bf245-b014-474c-85d8-3eb3fcbbb76c admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:12:41.077365 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6842f69c-1cb3-4e26-9b14-a6e377317041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:12:41.077958 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842f69c-1cb3-4e26-9b14-a6e377317041 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:12:41.078236 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842f69c-1cb3-4e26-9b14-a6e377317041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:12:41.078506 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842f69c-1cb3-4e26-9b14-a6e377317041 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:12:41.079139 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6842f69c-1cb3-4e26-9b14-a6e377317041 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:12:41.080748 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6842f69c-1cb3-4e26-9b14-a6e377317041 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:12:41.091735 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1273/2525] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:12:41.095601 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36248c4-9058-4fe0-a1a0-328ee167c396 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:12:41.106130 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36248c4-9058-4fe0-a1a0-328ee167c396 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:12:41.110967 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821436b0-27a3-4841-801a-b564a7712d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:12:41.111652 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821436b0-27a3-4841-801a-b564a7712d00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:12:41.111931 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821436b0-27a3-4841-801a-b564a7712d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:12:41.112201 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821436b0-27a3-4841-801a-b564a7712d00 admin admin] PATH_INFO: `/v3/projects/4d82031516ca4d5a8ba44a0c98a99881/users/3834d5ab253c456fa928663f8a1e84d5/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:12:41.127448 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821436b0-27a3-4841-801a-b564a7712d00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d82031516ca4d5a8ba44a0c98a99881, user_id=3834d5ab253c456fa928663f8a1e84d5, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:12:41.130117 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821436b0-27a3-4841-801a-b564a7712d00 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:12:41.142623 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1253/2526] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 15:12:41 2022] PUT /identity/v3/projects/4d82031516ca4d5a8ba44a0c98a99881/users/3834d5ab253c456fa928663f8a1e84d5/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:12:41.210040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa5c649-3f98-4f0b-b563-39323cca1674 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:12:41.210327 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa5c649-3f98-4f0b-b563-39323cca1674 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:12:41.210606 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa5c649-3f98-4f0b-b563-39323cca1674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:12:41.240045 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-eaa5c649-3f98-4f0b-b563-39323cca1674 None None] MFA Rules not processed for user `3834d5ab253c456fa928663f8a1e84d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:12:41.241261 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-eaa5c649-3f98-4f0b-b563-39323cca1674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:12:41.305516 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1274/2527] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:12:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:12:41.316678 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821436b0-27a3-4841-801a-b564a7712d00 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:12:41.318382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-821436b0-27a3-4841-801a-b564a7712d00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:12:41.407104 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821436b0-27a3-4841-801a-b564a7712d00 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:12:41.413540 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eff96b-3afe-41c8-a58d-81a373322a77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:12:41.414158 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eff96b-3afe-41c8-a58d-81a373322a77 service nova] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:12:41.415008 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eff96b-3afe-41c8-a58d-81a373322a77 service nova] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:12:41.415317 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eff96b-3afe-41c8-a58d-81a373322a77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:12:41.423548 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eff96b-3afe-41c8-a58d-81a373322a77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:12:41.425133 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eff96b-3afe-41c8-a58d-81a373322a77 service nova] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:12:41.434038 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1254/2528] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:12:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:12:51.302539 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=57,keystone:INSERT=3 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:12:51.430608 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=55,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:13:41.378238 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa5c649-3f98-4f0b-b563-39323cca1674 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:13:41.443822 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa5c649-3f98-4f0b-b563-39323cca1674 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:13:41.448458 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b053a-bf73-4104-8fc2-79b86a1f4416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:13:41.449397 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b053a-bf73-4104-8fc2-79b86a1f4416 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:13:41.449680 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b053a-bf73-4104-8fc2-79b86a1f4416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:13:41.449977 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b053a-bf73-4104-8fc2-79b86a1f4416 admin admin] PATH_INFO: `/v3/users/3834d5ab253c456fa928663f8a1e84d5` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:13:41.451166 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b053a-bf73-4104-8fc2-79b86a1f4416 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3834d5ab253c456fa928663f8a1e84d5)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:13:41.453159 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b053a-bf73-4104-8fc2-79b86a1f4416 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:13:41.498985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-186b053a-bf73-4104-8fc2-79b86a1f4416 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3834d5ab253c456fa928663f8a1e84d5'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:13:41.499582 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-186b053a-bf73-4104-8fc2-79b86a1f4416 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3834d5ab253c456fa928663f8a1e84d5'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:13:41.506694 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-186b053a-bf73-4104-8fc2-79b86a1f4416 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3834d5ab253c456fa928663f8a1e84d5'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:13:41.513890 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-186b053a-bf73-4104-8fc2-79b86a1f4416 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3834d5ab253c456fa928663f8a1e84d5'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:13:41.514532 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-186b053a-bf73-4104-8fc2-79b86a1f4416 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3834d5ab253c456fa928663f8a1e84d5'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:13:41.531867 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1275/2529] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 15:13:41 2022] DELETE /identity/v3/users/3834d5ab253c456fa928663f8a1e84d5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:13:41.536325 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611fa6cc-f1a3-4839-a1e4-f75c284f64ef None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:13:41.536818 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611fa6cc-f1a3-4839-a1e4-f75c284f64ef None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:13:41.536933 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611fa6cc-f1a3-4839-a1e4-f75c284f64ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:13:41.557406 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-611fa6cc-f1a3-4839-a1e4-f75c284f64ef None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:13:41.558624 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-611fa6cc-f1a3-4839-a1e4-f75c284f64ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:13:41.616049 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1255/2530] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:13:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:13:41.626410 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b053a-bf73-4104-8fc2-79b86a1f4416 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:13:41.627852 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-186b053a-bf73-4104-8fc2-79b86a1f4416 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:13:41.705981 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b053a-bf73-4104-8fc2-79b86a1f4416 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:13:41.709772 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d255c23-4f46-4304-b903-2f7e2cb0130f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:13:41.710959 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d255c23-4f46-4304-b903-2f7e2cb0130f service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:13:41.711264 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d255c23-4f46-4304-b903-2f7e2cb0130f service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:13:41.711542 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d255c23-4f46-4304-b903-2f7e2cb0130f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:13:41.720688 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d255c23-4f46-4304-b903-2f7e2cb0130f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:13:41.722106 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d255c23-4f46-4304-b903-2f7e2cb0130f service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:13:41.732426 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1276/2531] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:13:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:13:42.000002 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611fa6cc-f1a3-4839-a1e4-f75c284f64ef None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:13:42.001447 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-611fa6cc-f1a3-4839-a1e4-f75c284f64ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:13:42.014388 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611fa6cc-f1a3-4839-a1e4-f75c284f64ef None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:13:42.020726 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df675278-1541-4d0d-a030-1843560e989f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:13:42.021368 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df675278-1541-4d0d-a030-1843560e989f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:13:42.021720 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df675278-1541-4d0d-a030-1843560e989f admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:13:42.021913 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df675278-1541-4d0d-a030-1843560e989f admin admin] PATH_INFO: `/v3/projects/4d82031516ca4d5a8ba44a0c98a99881` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:13:42.023758 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df675278-1541-4d0d-a030-1843560e989f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d82031516ca4d5a8ba44a0c98a99881)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:13:42.026942 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df675278-1541-4d0d-a030-1843560e989f admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:13:42.076420 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-df675278-1541-4d0d-a030-1843560e989f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d82031516ca4d5a8ba44a0c98a99881', 'request_id': 'req-df675278-1541-4d0d-a030-1843560e989f'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:13:42.081953 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-df675278-1541-4d0d-a030-1843560e989f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d82031516ca4d5a8ba44a0c98a99881', 'request_id': 'req-df675278-1541-4d0d-a030-1843560e989f'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:13:42.089038 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-df675278-1541-4d0d-a030-1843560e989f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d82031516ca4d5a8ba44a0c98a99881', 'request_id': 'req-df675278-1541-4d0d-a030-1843560e989f'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:13:42.101248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-df675278-1541-4d0d-a030-1843560e989f admin admin] The token cache is being invalidate because project 4d82031516ca4d5a8ba44a0c98a99881 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:13:42.101537 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-df675278-1541-4d0d-a030-1843560e989f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:13:42.102160 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-df675278-1541-4d0d-a030-1843560e989f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:13:42.103696 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1256/2532] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 15:13:41 2022] DELETE /identity/v3/projects/4d82031516ca4d5a8ba44a0c98a99881 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:13:42.108862 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa4607-6f8e-450f-a6bd-36b5b6d18d2c None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:13:42.109177 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa4607-6f8e-450f-a6bd-36b5b6d18d2c None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:13:42.109450 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa4607-6f8e-450f-a6bd-36b5b6d18d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:13:42.117394 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:13:42.131498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-5eaa4607-6f8e-450f-a6bd-36b5b6d18d2c None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:13:42.133204 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-5eaa4607-6f8e-450f-a6bd-36b5b6d18d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:13:42.191168 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1277/2533] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:13:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:13:42.199726 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc938ee0-ee46-41ca-a37f-0386191d8a8d None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:13:42.200010 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc938ee0-ee46-41ca-a37f-0386191d8a8d None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:13:42.200281 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc938ee0-ee46-41ca-a37f-0386191d8a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:13:42.221712 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-bc938ee0-ee46-41ca-a37f-0386191d8a8d None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:13:42.222927 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-bc938ee0-ee46-41ca-a37f-0386191d8a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:13:42.231719 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1257/2534] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:13:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:13:42.235947 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa4607-6f8e-450f-a6bd-36b5b6d18d2c None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:13:42.246780 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa4607-6f8e-450f-a6bd-36b5b6d18d2c None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:13:42.251443 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bfec5b-1848-4f84-b351-024d5fbccd65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:13:42.252035 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bfec5b-1848-4f84-b351-024d5fbccd65 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:13:42.252310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bfec5b-1848-4f84-b351-024d5fbccd65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:13:42.252575 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bfec5b-1848-4f84-b351-024d5fbccd65 admin admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:13:42.253209 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bfec5b-1848-4f84-b351-024d5fbccd65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:13:42.253526 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bfec5b-1848-4f84-b351-024d5fbccd65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:13:42.255097 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bfec5b-1848-4f84-b351-024d5fbccd65 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:13:42.266920 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1278/2535] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 15:13:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:13:42.270964 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc938ee0-ee46-41ca-a37f-0386191d8a8d None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:13:42.280264 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc938ee0-ee46-41ca-a37f-0386191d8a8d None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:13:42.285163 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685c3c5b-d652-4019-9368-8123ce9184ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:13:42.285763 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c3c5b-d652-4019-9368-8123ce9184ee admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:13:42.286041 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c3c5b-d652-4019-9368-8123ce9184ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:13:42.286324 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c3c5b-d652-4019-9368-8123ce9184ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:13:42.286974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685c3c5b-d652-4019-9368-8123ce9184ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:13:42.288918 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685c3c5b-d652-4019-9368-8123ce9184ee admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:13:42.317517 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1258/2536] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 15:13:42 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:13:42.321813 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bfec5b-1848-4f84-b351-024d5fbccd65 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:13:42.332252 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bfec5b-1848-4f84-b351-024d5fbccd65 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:13:42.388602 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd53db-16f1-4790-ae80-8a8fa711f848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:13:42.389266 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd53db-16f1-4790-ae80-8a8fa711f848 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:13:42.389502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd53db-16f1-4790-ae80-8a8fa711f848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:13:42.389768 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd53db-16f1-4790-ae80-8a8fa711f848 admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:13:42.390430 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addd53db-16f1-4790-ae80-8a8fa711f848 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:13:42.392374 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addd53db-16f1-4790-ae80-8a8fa711f848 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:13:42.413888 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1279/2537] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 15:13:42 2022] POST /identity/v3/users => generated 285 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:13:42.418760 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685c3c5b-d652-4019-9368-8123ce9184ee admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:13:42.428941 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685c3c5b-d652-4019-9368-8123ce9184ee admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:13:42.434128 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45433a96-8213-4e3a-916e-58d38fc44d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:13:42.434720 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45433a96-8213-4e3a-916e-58d38fc44d09 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:13:42.435002 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45433a96-8213-4e3a-916e-58d38fc44d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:13:42.435314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45433a96-8213-4e3a-916e-58d38fc44d09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:13:42.435959 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45433a96-8213-4e3a-916e-58d38fc44d09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:13:42.437550 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45433a96-8213-4e3a-916e-58d38fc44d09 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:13:42.447957 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1259/2538] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 15:13:42.452067 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd53db-16f1-4790-ae80-8a8fa711f848 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:13:42.462675 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd53db-16f1-4790-ae80-8a8fa711f848 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:13:42.467359 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4335bca8-5898-47a3-982b-59261a139004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:13:42.467957 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4335bca8-5898-47a3-982b-59261a139004 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:13:42.468240 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4335bca8-5898-47a3-982b-59261a139004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:13:42.468516 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4335bca8-5898-47a3-982b-59261a139004 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:13:42.469164 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4335bca8-5898-47a3-982b-59261a139004 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:13:42.470744 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4335bca8-5898-47a3-982b-59261a139004 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:13:42.481920 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1280/2539] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:13:42.485919 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45433a96-8213-4e3a-916e-58d38fc44d09 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:13:42.496395 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45433a96-8213-4e3a-916e-58d38fc44d09 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:13:42.500982 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17ee54-51a3-4190-aca3-9a88532eecbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:13:42.501626 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17ee54-51a3-4190-aca3-9a88532eecbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:13:42.501906 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17ee54-51a3-4190-aca3-9a88532eecbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:13:42.502180 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17ee54-51a3-4190-aca3-9a88532eecbe admin admin] PATH_INFO: `/v3/projects/2ef7d7de806d452ca6d6b55ff313e654/users/29d6aecc4a6246daafc58620943f3ecb/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:13:42.517182 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad17ee54-51a3-4190-aca3-9a88532eecbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ef7d7de806d452ca6d6b55ff313e654, user_id=29d6aecc4a6246daafc58620943f3ecb, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:13:42.519875 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad17ee54-51a3-4190-aca3-9a88532eecbe admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:13:42.532858 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1260/2540] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 15:13:42 2022] PUT /identity/v3/projects/2ef7d7de806d452ca6d6b55ff313e654/users/29d6aecc4a6246daafc58620943f3ecb/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:13:42.542927 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e047da9-591d-466d-8b83-88d3a85204ba None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:13:42.543229 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e047da9-591d-466d-8b83-88d3a85204ba None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:13:42.543532 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e047da9-591d-466d-8b83-88d3a85204ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:13:42.571806 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-4e047da9-591d-466d-8b83-88d3a85204ba None None] MFA Rules not processed for user `29d6aecc4a6246daafc58620943f3ecb`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:13:42.573006 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-4e047da9-591d-466d-8b83-88d3a85204ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:13:42.630088 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:13:42.636683 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1281/2541] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:13:42 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:13:42.640968 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17ee54-51a3-4190-aca3-9a88532eecbe admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:13:42.661984 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:13:42.704470 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17ee54-51a3-4190-aca3-9a88532eecbe admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:13:42.709127 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3ed4c5-c6e2-42e3-b8b0-9551217ca8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:13:42.709723 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ed4c5-c6e2-42e3-b8b0-9551217ca8f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:13:42.710006 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ed4c5-c6e2-42e3-b8b0-9551217ca8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:13:42.710280 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ed4c5-c6e2-42e3-b8b0-9551217ca8f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:13:42.710921 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3ed4c5-c6e2-42e3-b8b0-9551217ca8f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:13:42.712849 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3ed4c5-c6e2-42e3-b8b0-9551217ca8f4 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:13:42.742948 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1261/2542] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 15:13:42 2022] POST /identity/v3/projects => generated 369 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:13:42.744210 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e047da9-591d-466d-8b83-88d3a85204ba None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:13:42.755072 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e047da9-591d-466d-8b83-88d3a85204ba None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:13:42.812315 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5185f1-20a6-4300-95f6-23cbcab1e234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:13:42.812952 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5185f1-20a6-4300-95f6-23cbcab1e234 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:13:42.813235 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5185f1-20a6-4300-95f6-23cbcab1e234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:13:42.813535 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5185f1-20a6-4300-95f6-23cbcab1e234 admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:13:42.814159 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5185f1-20a6-4300-95f6-23cbcab1e234 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:13:42.816386 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5185f1-20a6-4300-95f6-23cbcab1e234 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:13:42.836818 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1282/2543] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 15:13:42 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:13:42.840824 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3ed4c5-c6e2-42e3-b8b0-9551217ca8f4 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:13:42.851093 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3ed4c5-c6e2-42e3-b8b0-9551217ca8f4 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:13:42.856721 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f7fcc5-b6ad-432a-8a1c-4f106eae724e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:13:42.857318 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f7fcc5-b6ad-432a-8a1c-4f106eae724e admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:13:42.857596 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f7fcc5-b6ad-432a-8a1c-4f106eae724e admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:13:42.857867 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f7fcc5-b6ad-432a-8a1c-4f106eae724e admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:13:42.858506 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f7fcc5-b6ad-432a-8a1c-4f106eae724e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:13:42.860120 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f7fcc5-b6ad-432a-8a1c-4f106eae724e admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:13:42.871232 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1262/2544] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:13:42.875193 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5185f1-20a6-4300-95f6-23cbcab1e234 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:13:42.885887 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5185f1-20a6-4300-95f6-23cbcab1e234 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:13:42.890472 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f1d3b7-ca0f-495a-8871-c787103e2b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:13:42.891116 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f1d3b7-ca0f-495a-8871-c787103e2b43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:13:42.891428 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f1d3b7-ca0f-495a-8871-c787103e2b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:13:42.891705 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f1d3b7-ca0f-495a-8871-c787103e2b43 admin admin] PATH_INFO: `/v3/projects/d87fbf7f967049dc82e5342bd770b73d/users/f6ac190d41e04f9a8343a73c1dae1bbb/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:13:42.906931 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f1d3b7-ca0f-495a-8871-c787103e2b43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d87fbf7f967049dc82e5342bd770b73d, user_id=f6ac190d41e04f9a8343a73c1dae1bbb, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:13:42.909693 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f1d3b7-ca0f-495a-8871-c787103e2b43 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:13:42.922734 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1283/2545] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 15:13:42 2022] PUT /identity/v3/projects/d87fbf7f967049dc82e5342bd770b73d/users/f6ac190d41e04f9a8343a73c1dae1bbb/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:13:42.926675 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f7fcc5-b6ad-432a-8a1c-4f106eae724e admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:13:42.936597 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f7fcc5-b6ad-432a-8a1c-4f106eae724e admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:13:42.993177 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01125194-22a5-49ed-8d31-c34b53ddc117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:13:42.993802 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01125194-22a5-49ed-8d31-c34b53ddc117 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:13:42.994078 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01125194-22a5-49ed-8d31-c34b53ddc117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:13:42.994347 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01125194-22a5-49ed-8d31-c34b53ddc117 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:13:42.994980 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01125194-22a5-49ed-8d31-c34b53ddc117 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:13:42.996622 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01125194-22a5-49ed-8d31-c34b53ddc117 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:13:43.006072 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1263/2546] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:13:43.010755 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f1d3b7-ca0f-495a-8871-c787103e2b43 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:13:43.021086 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f1d3b7-ca0f-495a-8871-c787103e2b43 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:13:43.026373 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483e3dcf-f9a7-41aa-a60d-b281192becb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:13:43.027025 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e3dcf-f9a7-41aa-a60d-b281192becb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:13:43.027338 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e3dcf-f9a7-41aa-a60d-b281192becb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:13:43.027616 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e3dcf-f9a7-41aa-a60d-b281192becb8 admin admin] PATH_INFO: `/v3/projects/d87fbf7f967049dc82e5342bd770b73d/users/f6ac190d41e04f9a8343a73c1dae1bbb/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:13:43.029558 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483e3dcf-f9a7-41aa-a60d-b281192becb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d87fbf7f967049dc82e5342bd770b73d, user_id=f6ac190d41e04f9a8343a73c1dae1bbb, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:13:43.032250 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483e3dcf-f9a7-41aa-a60d-b281192becb8 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:13:43.042713 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1284/2547] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 15:13:43 2022] PUT /identity/v3/projects/d87fbf7f967049dc82e5342bd770b73d/users/f6ac190d41e04f9a8343a73c1dae1bbb/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:13:43.053742 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a487b83-ce45-469f-9c00-d869990d6f60 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:13:43.054027 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a487b83-ce45-469f-9c00-d869990d6f60 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:13:43.054300 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a487b83-ce45-469f-9c00-d869990d6f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:13:43.082178 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-8a487b83-ce45-469f-9c00-d869990d6f60 None None] MFA Rules not processed for user `f6ac190d41e04f9a8343a73c1dae1bbb`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:13:43.083419 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-8a487b83-ce45-469f-9c00-d869990d6f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:13:43.150075 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1264/2548] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:13:43 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:13:53.039648 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=45,keystone:INSERT=5 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:13:53.147110 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=92,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:14:37.329850 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483e3dcf-f9a7-41aa-a60d-b281192becb8 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:14:37.331731 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-483e3dcf-f9a7-41aa-a60d-b281192becb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:14:37.404707 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483e3dcf-f9a7-41aa-a60d-b281192becb8 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:14:37.410309 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd43cd7-aa89-435f-8b60-4f55fc7cc6a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:14:37.410933 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd43cd7-aa89-435f-8b60-4f55fc7cc6a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:14:37.412348 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd43cd7-aa89-435f-8b60-4f55fc7cc6a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:14:37.412629 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd43cd7-aa89-435f-8b60-4f55fc7cc6a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:14:37.415490 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-5bd43cd7-aa89-435f-8b60-4f55fc7cc6a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:14:37.480707 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd43cd7-aa89-435f-8b60-4f55fc7cc6a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:14:37.482067 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd43cd7-aa89-435f-8b60-4f55fc7cc6a3 service cinder] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:14:37.491513 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1285/2549] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:14:37 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:14:37.554238 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a487b83-ce45-469f-9c00-d869990d6f60 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:14:37.565725 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a487b83-ce45-469f-9c00-d869990d6f60 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:14:37.569792 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266a0f86-c0b2-42eb-97fe-f1b24d183f82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:14:37.571142 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266a0f86-c0b2-42eb-97fe-f1b24d183f82 service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:14:37.571551 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266a0f86-c0b2-42eb-97fe-f1b24d183f82 service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:14:37.571859 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266a0f86-c0b2-42eb-97fe-f1b24d183f82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:14:37.573614 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-266a0f86-c0b2-42eb-97fe-f1b24d183f82 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:14:37.671615 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266a0f86-c0b2-42eb-97fe-f1b24d183f82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:14:37.673086 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266a0f86-c0b2-42eb-97fe-f1b24d183f82 service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:14:37.684504 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1265/2550] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:14:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:14:37.814488 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd43cd7-aa89-435f-8b60-4f55fc7cc6a3 service cinder] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:14:37.823889 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd43cd7-aa89-435f-8b60-4f55fc7cc6a3 service cinder] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:14:37.828028 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f11cc1d-6615-4a56-9ff2-e2b627816e4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:14:37.828627 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f11cc1d-6615-4a56-9ff2-e2b627816e4c service cinder] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:14:37.828904 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f11cc1d-6615-4a56-9ff2-e2b627816e4c service cinder] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:14:37.829173 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f11cc1d-6615-4a56-9ff2-e2b627816e4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:14:37.838683 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f11cc1d-6615-4a56-9ff2-e2b627816e4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:14:37.840148 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f11cc1d-6615-4a56-9ff2-e2b627816e4c service cinder] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:14:37.850052 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1286/2551] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:14:37 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:14:47.679574 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=39 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:14:47.845482 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=59 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:15:54.244246 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266a0f86-c0b2-42eb-97fe-f1b24d183f82 service cinder] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:15:54.246000 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-266a0f86-c0b2-42eb-97fe-f1b24d183f82 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:15:54.301240 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266a0f86-c0b2-42eb-97fe-f1b24d183f82 service cinder] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:15:54.305001 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e15feb-f7b1-41a0-a812-192490515814 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:15:54.305601 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e15feb-f7b1-41a0-a812-192490515814 service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:15:54.305878 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e15feb-f7b1-41a0-a812-192490515814 service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:15:54.306148 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e15feb-f7b1-41a0-a812-192490515814 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:15:54.307857 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-62e15feb-f7b1-41a0-a812-192490515814 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:15:54.317225 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e15feb-f7b1-41a0-a812-192490515814 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:15:54.318628 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e15feb-f7b1-41a0-a812-192490515814 service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:15:54.327115 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1266/2552] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:15:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:15:57.594061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f11cc1d-6615-4a56-9ff2-e2b627816e4c service cinder] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:15:57.595574 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-9f11cc1d-6615-4a56-9ff2-e2b627816e4c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:15:57.663126 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f11cc1d-6615-4a56-9ff2-e2b627816e4c service cinder] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:15:57.668230 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a615b63f-cd83-461f-a002-e5c080a013f5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcf302a170934c25b39699088dac326a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:15:57.668825 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615b63f-cd83-461f-a002-e5c080a013f5 service placement] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:15:57.669103 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615b63f-cd83-461f-a002-e5c080a013f5 service placement] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:15:57.670076 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615b63f-cd83-461f-a002-e5c080a013f5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:15:57.671822 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-a615b63f-cd83-461f-a002-e5c080a013f5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:15:57.683299 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a615b63f-cd83-461f-a002-e5c080a013f5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:15:57.685438 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a615b63f-cd83-461f-a002-e5c080a013f5 service placement] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:15:57.695702 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1287/2553] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:15:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:16:04.322901 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=26 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:16:07.691452 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=32 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:16:38.102249 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e15feb-f7b1-41a0-a812-192490515814 service neutron] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:16:38.113674 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e15feb-f7b1-41a0-a812-192490515814 service neutron] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:16:38.117306 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a95f743-54d0-4d0c-ae3d-41a5ff3e3582 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:16:38.117898 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a95f743-54d0-4d0c-ae3d-41a5ff3e3582 service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:16:38.118178 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a95f743-54d0-4d0c-ae3d-41a5ff3e3582 service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:16:38.118452 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a95f743-54d0-4d0c-ae3d-41a5ff3e3582 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:16:38.120239 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-8a95f743-54d0-4d0c-ae3d-41a5ff3e3582 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:16:38.187807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a95f743-54d0-4d0c-ae3d-41a5ff3e3582 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:16:38.189276 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a95f743-54d0-4d0c-ae3d-41a5ff3e3582 service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:16:38.207830 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1267/2554] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:16:38 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:16:48.204777 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=36 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:20.790957 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a615b63f-cd83-461f-a002-e5c080a013f5 service placement] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:20.875016 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a615b63f-cd83-461f-a002-e5c080a013f5 service placement] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:20.879953 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd6f8ae-1528-44cf-97c6-67c18e733b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:20.880626 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6f8ae-1528-44cf-97c6-67c18e733b1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:20.880904 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6f8ae-1528-44cf-97c6-67c18e733b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:20.881174 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6f8ae-1528-44cf-97c6-67c18e733b1f admin admin] PATH_INFO: `/v3/users/29d6aecc4a6246daafc58620943f3ecb` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:20.882308 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd6f8ae-1528-44cf-97c6-67c18e733b1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=29d6aecc4a6246daafc58620943f3ecb)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:20.884261 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd6f8ae-1528-44cf-97c6-67c18e733b1f admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:20.940622 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-9dd6f8ae-1528-44cf-97c6-67c18e733b1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29d6aecc4a6246daafc58620943f3ecb'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:20.941309 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-9dd6f8ae-1528-44cf-97c6-67c18e733b1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29d6aecc4a6246daafc58620943f3ecb'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:20.949243 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-9dd6f8ae-1528-44cf-97c6-67c18e733b1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29d6aecc4a6246daafc58620943f3ecb'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:20.956978 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-9dd6f8ae-1528-44cf-97c6-67c18e733b1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29d6aecc4a6246daafc58620943f3ecb'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:20.957606 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-9dd6f8ae-1528-44cf-97c6-67c18e733b1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29d6aecc4a6246daafc58620943f3ecb'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:20.975290 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1288/2555] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 15:17:20 2022] DELETE /identity/v3/users/29d6aecc4a6246daafc58620943f3ecb => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:17:20.979710 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1c7dc-df29-4552-a4e6-6338b2f86461 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:20.980214 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1c7dc-df29-4552-a4e6-6338b2f86461 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:20.980328 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1c7dc-df29-4552-a4e6-6338b2f86461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:21.012569 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-def1c7dc-df29-4552-a4e6-6338b2f86461 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:17:21.013799 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-def1c7dc-df29-4552-a4e6-6338b2f86461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:21.074330 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1268/2556] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:17:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:17:21.086786 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd6f8ae-1528-44cf-97c6-67c18e733b1f admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:21.088233 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-9dd6f8ae-1528-44cf-97c6-67c18e733b1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:21.150576 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd6f8ae-1528-44cf-97c6-67c18e733b1f admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:21.154968 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2042b35c-f299-4e9c-bfb5-b18dece4483a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:21.155629 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042b35c-f299-4e9c-bfb5-b18dece4483a service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:21.155920 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042b35c-f299-4e9c-bfb5-b18dece4483a service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:21.156178 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042b35c-f299-4e9c-bfb5-b18dece4483a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:21.165578 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2042b35c-f299-4e9c-bfb5-b18dece4483a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:21.167009 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2042b35c-f299-4e9c-bfb5-b18dece4483a service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:21.179071 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1289/2557] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:17:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 15:17:21.495504 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def1c7dc-df29-4552-a4e6-6338b2f86461 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:21.497492 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-def1c7dc-df29-4552-a4e6-6338b2f86461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:21.514456 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def1c7dc-df29-4552-a4e6-6338b2f86461 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:21.519824 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2042b35c-f299-4e9c-bfb5-b18dece4483a service neutron] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:21.526146 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:21.527814 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff32cd70-647a-46c8-89a0-5341c556570b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:21.528819 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff32cd70-647a-46c8-89a0-5341c556570b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:21.529324 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff32cd70-647a-46c8-89a0-5341c556570b admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:21.529814 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff32cd70-647a-46c8-89a0-5341c556570b admin admin] PATH_INFO: `/v3/projects/2ef7d7de806d452ca6d6b55ff313e654` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:21.530338 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2042b35c-f299-4e9c-bfb5-b18dece4483a service neutron] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:21.531791 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff32cd70-647a-46c8-89a0-5341c556570b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ef7d7de806d452ca6d6b55ff313e654)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:21.534360 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff32cd70-647a-46c8-89a0-5341c556570b admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:21.536842 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92380001-7ca7-4193-aefe-85c8e8ae17e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:21.537605 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92380001-7ca7-4193-aefe-85c8e8ae17e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:21.537953 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92380001-7ca7-4193-aefe-85c8e8ae17e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:21.538486 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92380001-7ca7-4193-aefe-85c8e8ae17e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:21.541865 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-92380001-7ca7-4193-aefe-85c8e8ae17e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:21.594772 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-ff32cd70-647a-46c8-89a0-5341c556570b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ef7d7de806d452ca6d6b55ff313e654', 'request_id': 'req-ff32cd70-647a-46c8-89a0-5341c556570b'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:21.602326 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-ff32cd70-647a-46c8-89a0-5341c556570b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ef7d7de806d452ca6d6b55ff313e654', 'request_id': 'req-ff32cd70-647a-46c8-89a0-5341c556570b'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:21.611124 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-ff32cd70-647a-46c8-89a0-5341c556570b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ef7d7de806d452ca6d6b55ff313e654', 'request_id': 'req-ff32cd70-647a-46c8-89a0-5341c556570b'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:21.629230 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-ff32cd70-647a-46c8-89a0-5341c556570b admin admin] The token cache is being invalidate because project 2ef7d7de806d452ca6d6b55ff313e654 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:17:21.629695 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-ff32cd70-647a-46c8-89a0-5341c556570b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:21.630280 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-ff32cd70-647a-46c8-89a0-5341c556570b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:21.633056 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1269/2558] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 15:17:21 2022] DELETE /identity/v3/projects/2ef7d7de806d452ca6d6b55ff313e654 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:17:21.636719 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff32cd70-647a-46c8-89a0-5341c556570b admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:21.638124 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92380001-7ca7-4193-aefe-85c8e8ae17e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:21.638316 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-ff32cd70-647a-46c8-89a0-5341c556570b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:21.640319 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92380001-7ca7-4193-aefe-85c8e8ae17e5 service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:21.656909 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1290/2559] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:17:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 15:17:21.719886 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff32cd70-647a-46c8-89a0-5341c556570b admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:21.725795 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0801382-9d05-4900-a3fe-3f2589ad3d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:21.726523 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0801382-9d05-4900-a3fe-3f2589ad3d10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:21.726880 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0801382-9d05-4900-a3fe-3f2589ad3d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:21.727132 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0801382-9d05-4900-a3fe-3f2589ad3d10 admin admin] PATH_INFO: `/v3/users/f6ac190d41e04f9a8343a73c1dae1bbb` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:21.728504 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0801382-9d05-4900-a3fe-3f2589ad3d10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6ac190d41e04f9a8343a73c1dae1bbb)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:21.730453 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0801382-9d05-4900-a3fe-3f2589ad3d10 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:21.757315 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:21.810387 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-f0801382-9d05-4900-a3fe-3f2589ad3d10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6ac190d41e04f9a8343a73c1dae1bbb'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:21.811345 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-f0801382-9d05-4900-a3fe-3f2589ad3d10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6ac190d41e04f9a8343a73c1dae1bbb'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:21.833208 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-f0801382-9d05-4900-a3fe-3f2589ad3d10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6ac190d41e04f9a8343a73c1dae1bbb'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:21.841173 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-f0801382-9d05-4900-a3fe-3f2589ad3d10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6ac190d41e04f9a8343a73c1dae1bbb'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:21.849306 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-f0801382-9d05-4900-a3fe-3f2589ad3d10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6ac190d41e04f9a8343a73c1dae1bbb'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:21.855696 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1270/2560] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 15:17:21 2022] DELETE /identity/v3/users/f6ac190d41e04f9a8343a73c1dae1bbb => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:17:21.938205 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92380001-7ca7-4193-aefe-85c8e8ae17e5 service neutron] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:21.940342 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-92380001-7ca7-4193-aefe-85c8e8ae17e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:21.967824 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92380001-7ca7-4193-aefe-85c8e8ae17e5 service neutron] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:22.072786 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326ecf77-d2d5-44bb-90cf-b724d765a8e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:22.073547 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326ecf77-d2d5-44bb-90cf-b724d765a8e2 service nova] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:22.073833 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326ecf77-d2d5-44bb-90cf-b724d765a8e2 service nova] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:22.074122 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326ecf77-d2d5-44bb-90cf-b724d765a8e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:22.076877 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-326ecf77-d2d5-44bb-90cf-b724d765a8e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:22.091330 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326ecf77-d2d5-44bb-90cf-b724d765a8e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:22.093614 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326ecf77-d2d5-44bb-90cf-b724d765a8e2 service nova] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:22.113764 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1291/2561] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:17:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:17:22.274926 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0801382-9d05-4900-a3fe-3f2589ad3d10 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:22.276773 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-f0801382-9d05-4900-a3fe-3f2589ad3d10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:22.367061 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0801382-9d05-4900-a3fe-3f2589ad3d10 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:22.372222 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aadec6b-43c5-4f84-b7aa-1abc9560ec1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:22.373388 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aadec6b-43c5-4f84-b7aa-1abc9560ec1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:22.373683 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aadec6b-43c5-4f84-b7aa-1abc9560ec1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:22.373976 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aadec6b-43c5-4f84-b7aa-1abc9560ec1e admin admin] PATH_INFO: `/v3/projects/d87fbf7f967049dc82e5342bd770b73d` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:22.375155 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aadec6b-43c5-4f84-b7aa-1abc9560ec1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d87fbf7f967049dc82e5342bd770b73d)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:22.377131 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aadec6b-43c5-4f84-b7aa-1abc9560ec1e admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:22.439317 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-9aadec6b-43c5-4f84-b7aa-1abc9560ec1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd87fbf7f967049dc82e5342bd770b73d', 'request_id': 'req-9aadec6b-43c5-4f84-b7aa-1abc9560ec1e'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:22.446151 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-9aadec6b-43c5-4f84-b7aa-1abc9560ec1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd87fbf7f967049dc82e5342bd770b73d', 'request_id': 'req-9aadec6b-43c5-4f84-b7aa-1abc9560ec1e'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:22.454385 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-9aadec6b-43c5-4f84-b7aa-1abc9560ec1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd87fbf7f967049dc82e5342bd770b73d', 'request_id': 'req-9aadec6b-43c5-4f84-b7aa-1abc9560ec1e'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:22.483714 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-9aadec6b-43c5-4f84-b7aa-1abc9560ec1e admin admin] The token cache is being invalidate because project d87fbf7f967049dc82e5342bd770b73d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:17:22.484230 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-9aadec6b-43c5-4f84-b7aa-1abc9560ec1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:22.485358 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-9aadec6b-43c5-4f84-b7aa-1abc9560ec1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:22.490754 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1271/2562] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 15:17:22 2022] DELETE /identity/v3/projects/d87fbf7f967049dc82e5342bd770b73d => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:17:22.493855 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22635d9c-69fd-4413-888b-4b31ffdbd286 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:22.494145 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22635d9c-69fd-4413-888b-4b31ffdbd286 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:22.494500 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22635d9c-69fd-4413-888b-4b31ffdbd286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:22.608071 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-22635d9c-69fd-4413-888b-4b31ffdbd286 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:17:22.609828 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-22635d9c-69fd-4413-888b-4b31ffdbd286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:22.680418 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:22.766912 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1292/2563] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:17:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:17:22.775044 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab5e68c-9d13-4d6b-92e2-5a6eb695aac7 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:22.775974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab5e68c-9d13-4d6b-92e2-5a6eb695aac7 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:22.775974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab5e68c-9d13-4d6b-92e2-5a6eb695aac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:22.813297 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-1ab5e68c-9d13-4d6b-92e2-5a6eb695aac7 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:17:22.815047 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-1ab5e68c-9d13-4d6b-92e2-5a6eb695aac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:22.831600 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1272/2564] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:17:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:17:22.833560 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22635d9c-69fd-4413-888b-4b31ffdbd286 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:22.860048 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22635d9c-69fd-4413-888b-4b31ffdbd286 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:22.870624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cef0fc-66bb-489d-b29d-10cf449ce882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:22.871428 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cef0fc-66bb-489d-b29d-10cf449ce882 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:22.873291 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cef0fc-66bb-489d-b29d-10cf449ce882 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:22.874454 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cef0fc-66bb-489d-b29d-10cf449ce882 admin admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:22.875245 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cef0fc-66bb-489d-b29d-10cf449ce882 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:17:22.875648 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cef0fc-66bb-489d-b29d-10cf449ce882 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:22.877585 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cef0fc-66bb-489d-b29d-10cf449ce882 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:22.894545 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1293/2565] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 15:17:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:17:22.894877 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab5e68c-9d13-4d6b-92e2-5a6eb695aac7 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:22.906721 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab5e68c-9d13-4d6b-92e2-5a6eb695aac7 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:22.939800 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413dfd98-2804-4d08-b382-3ed3f1ab667c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:22.940858 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413dfd98-2804-4d08-b382-3ed3f1ab667c admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:22.941126 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413dfd98-2804-4d08-b382-3ed3f1ab667c admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:22.941423 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413dfd98-2804-4d08-b382-3ed3f1ab667c admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:22.942129 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413dfd98-2804-4d08-b382-3ed3f1ab667c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:22.944430 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413dfd98-2804-4d08-b382-3ed3f1ab667c admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:22.965271 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:22.995246 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1273/2566] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 15:17:22 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:17:22.999453 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cef0fc-66bb-489d-b29d-10cf449ce882 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:23.009821 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cef0fc-66bb-489d-b29d-10cf449ce882 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:23.080577 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d2043-15e8-46e8-845e-589bcbc7c5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:23.081352 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d2043-15e8-46e8-845e-589bcbc7c5ef admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:23.081737 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d2043-15e8-46e8-845e-589bcbc7c5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:23.082111 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d2043-15e8-46e8-845e-589bcbc7c5ef admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:23.082880 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30d2043-15e8-46e8-845e-589bcbc7c5ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:23.085024 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30d2043-15e8-46e8-845e-589bcbc7c5ef admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:23.110301 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1294/2567] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 15:17:22 2022] POST /identity/v3/users => generated 294 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:17:23.114449 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413dfd98-2804-4d08-b382-3ed3f1ab667c admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:23.128953 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413dfd98-2804-4d08-b382-3ed3f1ab667c admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:23.136560 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc6a39-4825-4909-b3a1-b632380a6a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:23.137224 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc6a39-4825-4909-b3a1-b632380a6a01 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:23.137513 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc6a39-4825-4909-b3a1-b632380a6a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:23.137864 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc6a39-4825-4909-b3a1-b632380a6a01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:23.138477 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fc6a39-4825-4909-b3a1-b632380a6a01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:23.140458 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fc6a39-4825-4909-b3a1-b632380a6a01 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:23.156352 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1274/2568] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:17:23 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:17:23.157007 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d2043-15e8-46e8-845e-589bcbc7c5ef admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:23.173289 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d2043-15e8-46e8-845e-589bcbc7c5ef admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:23.180133 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc234ae-fa52-4b62-b5ce-27455937469a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:23.180930 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc234ae-fa52-4b62-b5ce-27455937469a admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:23.181401 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc234ae-fa52-4b62-b5ce-27455937469a admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:23.181866 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc234ae-fa52-4b62-b5ce-27455937469a admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:23.182614 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc234ae-fa52-4b62-b5ce-27455937469a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:23.184470 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc234ae-fa52-4b62-b5ce-27455937469a admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:23.206065 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1295/2569] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:17:23 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:17:23.211238 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc6a39-4825-4909-b3a1-b632380a6a01 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:23.231638 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc6a39-4825-4909-b3a1-b632380a6a01 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:23.237559 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d90b3d-ca14-4ed7-a0f6-a05fcbcf170c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:23.238403 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d90b3d-ca14-4ed7-a0f6-a05fcbcf170c admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:23.238786 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d90b3d-ca14-4ed7-a0f6-a05fcbcf170c admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:23.239363 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d90b3d-ca14-4ed7-a0f6-a05fcbcf170c admin admin] PATH_INFO: `/v3/projects/4ea27d093e3a4a4faa38fb35f3b96e16/users/143c04c0f99f45bfa919c1000d4c5369/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:23.257550 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d90b3d-ca14-4ed7-a0f6-a05fcbcf170c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ea27d093e3a4a4faa38fb35f3b96e16, user_id=143c04c0f99f45bfa919c1000d4c5369, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:23.260505 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d90b3d-ca14-4ed7-a0f6-a05fcbcf170c admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:23.275816 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1275/2570] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 15:17:23 2022] PUT /identity/v3/projects/4ea27d093e3a4a4faa38fb35f3b96e16/users/143c04c0f99f45bfa919c1000d4c5369/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:17:23.286452 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f67353-1499-4204-81e7-8f4c58fb944b None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:23.287043 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f67353-1499-4204-81e7-8f4c58fb944b None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:23.287043 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f67353-1499-4204-81e7-8f4c58fb944b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:23.321633 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-49f67353-1499-4204-81e7-8f4c58fb944b None None] MFA Rules not processed for user `143c04c0f99f45bfa919c1000d4c5369`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:17:23.323486 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-49f67353-1499-4204-81e7-8f4c58fb944b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:23.396813 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1296/2571] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:17:23 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:17:23.401228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d90b3d-ca14-4ed7-a0f6-a05fcbcf170c admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:23.464894 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d90b3d-ca14-4ed7-a0f6-a05fcbcf170c admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:23.471556 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b344ae07-e407-4c2c-a80e-e87634e1560d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:23.472163 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344ae07-e407-4c2c-a80e-e87634e1560d admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:23.472441 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344ae07-e407-4c2c-a80e-e87634e1560d admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:23.472710 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344ae07-e407-4c2c-a80e-e87634e1560d admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:23.473342 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b344ae07-e407-4c2c-a80e-e87634e1560d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:23.475258 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b344ae07-e407-4c2c-a80e-e87634e1560d admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:23.506280 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1276/2572] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 15:17:23 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:17:23.507585 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f67353-1499-4204-81e7-8f4c58fb944b None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:23.519175 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f67353-1499-4204-81e7-8f4c58fb944b None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:23.561681 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:23.582567 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8e4575-ddc2-43a6-a9f5-868f3180a194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:23.583186 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e4575-ddc2-43a6-a9f5-868f3180a194 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:23.583493 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e4575-ddc2-43a6-a9f5-868f3180a194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:23.583766 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e4575-ddc2-43a6-a9f5-868f3180a194 admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:23.584406 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8e4575-ddc2-43a6-a9f5-868f3180a194 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:23.586337 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8e4575-ddc2-43a6-a9f5-868f3180a194 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:23.609999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1297/2573] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 15:17:23 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:17:23.614022 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b344ae07-e407-4c2c-a80e-e87634e1560d admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:23.624584 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b344ae07-e407-4c2c-a80e-e87634e1560d admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:23.629469 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8716a29a-e1fa-4c2c-9d84-68639da5b64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:23.630147 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8716a29a-e1fa-4c2c-9d84-68639da5b64a admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:23.630462 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8716a29a-e1fa-4c2c-9d84-68639da5b64a admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:23.630757 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8716a29a-e1fa-4c2c-9d84-68639da5b64a admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:23.631451 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8716a29a-e1fa-4c2c-9d84-68639da5b64a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:23.633163 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8716a29a-e1fa-4c2c-9d84-68639da5b64a admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:23.645244 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1277/2574] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:17:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:17:23.649733 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8e4575-ddc2-43a6-a9f5-868f3180a194 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:23.661467 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8e4575-ddc2-43a6-a9f5-868f3180a194 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:23.666321 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00706bc9-a135-4340-9adb-bd376e91d14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:23.666982 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00706bc9-a135-4340-9adb-bd376e91d14c admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:23.667296 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00706bc9-a135-4340-9adb-bd376e91d14c admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:23.667571 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00706bc9-a135-4340-9adb-bd376e91d14c admin admin] PATH_INFO: `/v3/projects/fcea289751c34876ba80adb36c839d2f/users/099c618147324f16b7c8be82a4cbcbf5/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:23.683454 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00706bc9-a135-4340-9adb-bd376e91d14c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcea289751c34876ba80adb36c839d2f, user_id=099c618147324f16b7c8be82a4cbcbf5, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:23.687011 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00706bc9-a135-4340-9adb-bd376e91d14c admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:23.701333 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1298/2575] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 15:17:23 2022] PUT /identity/v3/projects/fcea289751c34876ba80adb36c839d2f/users/099c618147324f16b7c8be82a4cbcbf5/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:17:23.705412 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8716a29a-e1fa-4c2c-9d84-68639da5b64a admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:23.715796 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8716a29a-e1fa-4c2c-9d84-68639da5b64a admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:23.777680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e75e3fc-5ba0-41af-a700-965be14bbbbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:23.778288 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e75e3fc-5ba0-41af-a700-965be14bbbbe admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:23.778567 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e75e3fc-5ba0-41af-a700-965be14bbbbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:23.778840 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e75e3fc-5ba0-41af-a700-965be14bbbbe admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:23.780034 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e75e3fc-5ba0-41af-a700-965be14bbbbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:23.781731 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e75e3fc-5ba0-41af-a700-965be14bbbbe admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:23.791888 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1278/2576] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:17:23 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:17:23.796569 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00706bc9-a135-4340-9adb-bd376e91d14c admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:23.807123 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00706bc9-a135-4340-9adb-bd376e91d14c admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:23.811722 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf0f46-45f3-4ca2-9b14-65a16b3509f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:23.812419 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf0f46-45f3-4ca2-9b14-65a16b3509f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:23.812640 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf0f46-45f3-4ca2-9b14-65a16b3509f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:23.812915 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf0f46-45f3-4ca2-9b14-65a16b3509f4 admin admin] PATH_INFO: `/v3/projects/fcea289751c34876ba80adb36c839d2f/users/099c618147324f16b7c8be82a4cbcbf5/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:23.814855 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacf0f46-45f3-4ca2-9b14-65a16b3509f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcea289751c34876ba80adb36c839d2f, user_id=099c618147324f16b7c8be82a4cbcbf5, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:23.817536 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacf0f46-45f3-4ca2-9b14-65a16b3509f4 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:23.829023 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1299/2577] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 15:17:23 2022] PUT /identity/v3/projects/fcea289751c34876ba80adb36c839d2f/users/099c618147324f16b7c8be82a4cbcbf5/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:17:23.840073 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7510b82-6d6a-41dc-96c8-c28f48561215 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:23.840433 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7510b82-6d6a-41dc-96c8-c28f48561215 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:23.840710 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7510b82-6d6a-41dc-96c8-c28f48561215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:23.874202 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-e7510b82-6d6a-41dc-96c8-c28f48561215 None None] MFA Rules not processed for user `099c618147324f16b7c8be82a4cbcbf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:17:23.875590 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-e7510b82-6d6a-41dc-96c8-c28f48561215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:23.884921 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:23.947437 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1279/2578] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:17:23 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:17:23.958891 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf0f46-45f3-4ca2-9b14-65a16b3509f4 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:23.961016 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-bacf0f46-45f3-4ca2-9b14-65a16b3509f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:24.020784 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf0f46-45f3-4ca2-9b14-65a16b3509f4 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:24.024441 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce251785-110c-4bd8-89c5-349c778770fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:24.025040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce251785-110c-4bd8-89c5-349c778770fd service cinder] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:24.025317 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce251785-110c-4bd8-89c5-349c778770fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:24.025586 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce251785-110c-4bd8-89c5-349c778770fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:24.036370 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce251785-110c-4bd8-89c5-349c778770fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:24.037812 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce251785-110c-4bd8-89c5-349c778770fd service cinder] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:24.050074 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1300/2579] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:17:23 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:17:24.119467 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7510b82-6d6a-41dc-96c8-c28f48561215 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:24.182186 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7510b82-6d6a-41dc-96c8-c28f48561215 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:24.187050 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ca433-0bbf-4795-9851-6bdf16adc756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:24.187724 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ca433-0bbf-4795-9851-6bdf16adc756 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:24.188005 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ca433-0bbf-4795-9851-6bdf16adc756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:24.188276 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ca433-0bbf-4795-9851-6bdf16adc756 admin admin] PATH_INFO: `/v3/users/143c04c0f99f45bfa919c1000d4c5369` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:24.189397 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85ca433-0bbf-4795-9851-6bdf16adc756 admin admin] RBAC: Authorizing `identity:delete_user(user_id=143c04c0f99f45bfa919c1000d4c5369)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:24.191348 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85ca433-0bbf-4795-9851-6bdf16adc756 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:24.239975 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-d85ca433-0bbf-4795-9851-6bdf16adc756 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '143c04c0f99f45bfa919c1000d4c5369'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:24.240552 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-d85ca433-0bbf-4795-9851-6bdf16adc756 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '143c04c0f99f45bfa919c1000d4c5369'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:24.253106 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-d85ca433-0bbf-4795-9851-6bdf16adc756 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '143c04c0f99f45bfa919c1000d4c5369'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:24.260079 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-d85ca433-0bbf-4795-9851-6bdf16adc756 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '143c04c0f99f45bfa919c1000d4c5369'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:24.268312 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-d85ca433-0bbf-4795-9851-6bdf16adc756 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '143c04c0f99f45bfa919c1000d4c5369'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:24.274240 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1280/2580] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 15:17:24 2022] DELETE /identity/v3/users/143c04c0f99f45bfa919c1000d4c5369 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:17:24.278760 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da9afd-8719-4ff7-a697-00c18b0559b1 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:24.279885 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da9afd-8719-4ff7-a697-00c18b0559b1 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:24.280270 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da9afd-8719-4ff7-a697-00c18b0559b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:24.298729 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-00da9afd-8719-4ff7-a697-00c18b0559b1 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:17:24.299962 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-00da9afd-8719-4ff7-a697-00c18b0559b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:24.360069 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1301/2581] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:17:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:17:24.369744 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ca433-0bbf-4795-9851-6bdf16adc756 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:24.371255 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-d85ca433-0bbf-4795-9851-6bdf16adc756 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:24.400223 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:24.432869 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ca433-0bbf-4795-9851-6bdf16adc756 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:24.437090 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b156e9-011d-4159-990f-ef0078097d09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:24.437708 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b156e9-011d-4159-990f-ef0078097d09 service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:24.437988 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b156e9-011d-4159-990f-ef0078097d09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:24.438258 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b156e9-011d-4159-990f-ef0078097d09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:24.447726 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b156e9-011d-4159-990f-ef0078097d09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:24.449164 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b156e9-011d-4159-990f-ef0078097d09 service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:24.459937 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1281/2582] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:17:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:17:24.767444 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da9afd-8719-4ff7-a697-00c18b0559b1 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:24.768900 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-00da9afd-8719-4ff7-a697-00c18b0559b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:24.784212 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da9afd-8719-4ff7-a697-00c18b0559b1 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:24.789004 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f08a7-0ec8-4075-8faf-ca0df9f4ea81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:24.789642 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f08a7-0ec8-4075-8faf-ca0df9f4ea81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:24.789922 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f08a7-0ec8-4075-8faf-ca0df9f4ea81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:24.790192 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f08a7-0ec8-4075-8faf-ca0df9f4ea81 admin admin] PATH_INFO: `/v3/projects/4ea27d093e3a4a4faa38fb35f3b96e16` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:24.791368 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02f08a7-0ec8-4075-8faf-ca0df9f4ea81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ea27d093e3a4a4faa38fb35f3b96e16)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:24.793269 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02f08a7-0ec8-4075-8faf-ca0df9f4ea81 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:24.829919 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:24.844417 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-d02f08a7-0ec8-4075-8faf-ca0df9f4ea81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ea27d093e3a4a4faa38fb35f3b96e16', 'request_id': 'req-d02f08a7-0ec8-4075-8faf-ca0df9f4ea81'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:24.850011 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-d02f08a7-0ec8-4075-8faf-ca0df9f4ea81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ea27d093e3a4a4faa38fb35f3b96e16', 'request_id': 'req-d02f08a7-0ec8-4075-8faf-ca0df9f4ea81'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:24.856056 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-d02f08a7-0ec8-4075-8faf-ca0df9f4ea81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ea27d093e3a4a4faa38fb35f3b96e16', 'request_id': 'req-d02f08a7-0ec8-4075-8faf-ca0df9f4ea81'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:24.868828 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-d02f08a7-0ec8-4075-8faf-ca0df9f4ea81 admin admin] The token cache is being invalidate because project 4ea27d093e3a4a4faa38fb35f3b96e16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:17:24.869115 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-d02f08a7-0ec8-4075-8faf-ca0df9f4ea81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:24.869730 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-d02f08a7-0ec8-4075-8faf-ca0df9f4ea81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:24.871225 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1302/2583] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 15:17:24 2022] DELETE /identity/v3/projects/4ea27d093e3a4a4faa38fb35f3b96e16 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:17:24.876265 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b156e9-011d-4159-990f-ef0078097d09 service neutron] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:24.885829 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b156e9-011d-4159-990f-ef0078097d09 service neutron] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:24.943016 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24a57d8-6d43-4931-b913-f8f3fe819002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:24.943688 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a57d8-6d43-4931-b913-f8f3fe819002 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:24.943973 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a57d8-6d43-4931-b913-f8f3fe819002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:24.944255 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a57d8-6d43-4931-b913-f8f3fe819002 admin admin] PATH_INFO: `/v3/users/099c618147324f16b7c8be82a4cbcbf5` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:24.945385 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24a57d8-6d43-4931-b913-f8f3fe819002 admin admin] RBAC: Authorizing `identity:delete_user(user_id=099c618147324f16b7c8be82a4cbcbf5)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:24.947328 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24a57d8-6d43-4931-b913-f8f3fe819002 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:24.993323 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b24a57d8-6d43-4931-b913-f8f3fe819002 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '099c618147324f16b7c8be82a4cbcbf5'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:24.993970 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b24a57d8-6d43-4931-b913-f8f3fe819002 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '099c618147324f16b7c8be82a4cbcbf5'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:25.005778 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b24a57d8-6d43-4931-b913-f8f3fe819002 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '099c618147324f16b7c8be82a4cbcbf5'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:25.012846 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b24a57d8-6d43-4931-b913-f8f3fe819002 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '099c618147324f16b7c8be82a4cbcbf5'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:25.020722 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b24a57d8-6d43-4931-b913-f8f3fe819002 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '099c618147324f16b7c8be82a4cbcbf5'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:25.026547 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1282/2584] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 15:17:24 2022] DELETE /identity/v3/users/099c618147324f16b7c8be82a4cbcbf5 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:17:25.308991 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f08a7-0ec8-4075-8faf-ca0df9f4ea81 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:25.310777 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-d02f08a7-0ec8-4075-8faf-ca0df9f4ea81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:25.326496 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f08a7-0ec8-4075-8faf-ca0df9f4ea81 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:25.385405 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93195d6f-6321-4494-9fef-2e22da7287da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:25.386036 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93195d6f-6321-4494-9fef-2e22da7287da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:25.386317 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93195d6f-6321-4494-9fef-2e22da7287da admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:25.386590 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93195d6f-6321-4494-9fef-2e22da7287da admin admin] PATH_INFO: `/v3/projects/fcea289751c34876ba80adb36c839d2f` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:25.387742 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93195d6f-6321-4494-9fef-2e22da7287da admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcea289751c34876ba80adb36c839d2f)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:25.389664 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93195d6f-6321-4494-9fef-2e22da7287da admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:25.435343 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-93195d6f-6321-4494-9fef-2e22da7287da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcea289751c34876ba80adb36c839d2f', 'request_id': 'req-93195d6f-6321-4494-9fef-2e22da7287da'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:25.441261 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-93195d6f-6321-4494-9fef-2e22da7287da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcea289751c34876ba80adb36c839d2f', 'request_id': 'req-93195d6f-6321-4494-9fef-2e22da7287da'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:25.447599 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-93195d6f-6321-4494-9fef-2e22da7287da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcea289751c34876ba80adb36c839d2f', 'request_id': 'req-93195d6f-6321-4494-9fef-2e22da7287da'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:25.460187 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-93195d6f-6321-4494-9fef-2e22da7287da admin admin] The token cache is being invalidate because project fcea289751c34876ba80adb36c839d2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:17:25.460475 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-93195d6f-6321-4494-9fef-2e22da7287da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:25.461097 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-93195d6f-6321-4494-9fef-2e22da7287da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:25.462596 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1303/2585] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 15:17:25 2022] DELETE /identity/v3/projects/fcea289751c34876ba80adb36c839d2f => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:17:25.468169 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5db96-87c2-4413-8da0-7b1f8a8d27a7 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:25.468648 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5db96-87c2-4413-8da0-7b1f8a8d27a7 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:25.468852 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5db96-87c2-4413-8da0-7b1f8a8d27a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:25.489537 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-afc5db96-87c2-4413-8da0-7b1f8a8d27a7 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:17:25.490744 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-afc5db96-87c2-4413-8da0-7b1f8a8d27a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:25.531593 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:25.550714 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1283/2586] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:17:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:17:25.558271 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c403ae-3cb4-4ccd-b1fe-e083391e08db None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:25.558589 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c403ae-3cb4-4ccd-b1fe-e083391e08db None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:25.558863 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c403ae-3cb4-4ccd-b1fe-e083391e08db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:25.583833 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-f3c403ae-3cb4-4ccd-b1fe-e083391e08db None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:17:25.585104 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-f3c403ae-3cb4-4ccd-b1fe-e083391e08db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:25.594971 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1304/2587] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:17:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:17:25.599427 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc5db96-87c2-4413-8da0-7b1f8a8d27a7 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:25.610220 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc5db96-87c2-4413-8da0-7b1f8a8d27a7 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:25.618507 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d6ce3-62c2-4fd5-b922-aeff7c40faff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:25.619116 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d6ce3-62c2-4fd5-b922-aeff7c40faff admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:25.619444 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d6ce3-62c2-4fd5-b922-aeff7c40faff admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:25.619716 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d6ce3-62c2-4fd5-b922-aeff7c40faff admin admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:25.620399 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5d6ce3-62c2-4fd5-b922-aeff7c40faff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:17:25.620700 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5d6ce3-62c2-4fd5-b922-aeff7c40faff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:25.622325 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5d6ce3-62c2-4fd5-b922-aeff7c40faff admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:25.639368 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1284/2588] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 15:17:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:17:25.643985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c403ae-3cb4-4ccd-b1fe-e083391e08db None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:25.655302 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c403ae-3cb4-4ccd-b1fe-e083391e08db None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:25.661281 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee1f45-a0e7-466d-8855-c3acefe29524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:25.662200 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee1f45-a0e7-466d-8855-c3acefe29524 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:25.662417 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee1f45-a0e7-466d-8855-c3acefe29524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:25.662826 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee1f45-a0e7-466d-8855-c3acefe29524 admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:25.663772 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fee1f45-a0e7-466d-8855-c3acefe29524 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:25.665868 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fee1f45-a0e7-466d-8855-c3acefe29524 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:25.696775 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1305/2589] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 15:17:25 2022] POST /identity/v3/projects => generated 407 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:17:25.701344 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d6ce3-62c2-4fd5-b922-aeff7c40faff admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:25.711488 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d6ce3-62c2-4fd5-b922-aeff7c40faff admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:25.771754 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54a3a2c-5f55-48a3-9d5c-4de305499338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:25.772358 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a3a2c-5f55-48a3-9d5c-4de305499338 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:25.772639 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a3a2c-5f55-48a3-9d5c-4de305499338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:25.772912 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a3a2c-5f55-48a3-9d5c-4de305499338 admin admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:25.773539 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54a3a2c-5f55-48a3-9d5c-4de305499338 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:25.775519 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54a3a2c-5f55-48a3-9d5c-4de305499338 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:25.797190 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1285/2590] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 15:17:25 2022] POST /identity/v3/users => generated 304 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:17:25.801380 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee1f45-a0e7-466d-8855-c3acefe29524 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:25.812465 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee1f45-a0e7-466d-8855-c3acefe29524 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:25.817094 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e66d26-8568-4ade-b86d-ea0316a471ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:25.817688 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e66d26-8568-4ade-b86d-ea0316a471ac admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:25.817965 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e66d26-8568-4ade-b86d-ea0316a471ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:25.818235 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e66d26-8568-4ade-b86d-ea0316a471ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:25.818877 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e66d26-8568-4ade-b86d-ea0316a471ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:25.820496 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e66d26-8568-4ade-b86d-ea0316a471ac admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:25.831199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1306/2591] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:17:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:17:25.835247 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54a3a2c-5f55-48a3-9d5c-4de305499338 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:25.845299 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54a3a2c-5f55-48a3-9d5c-4de305499338 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:25.849884 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2a7c46-3c7d-43e9-8d5c-f028a6e3b9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:25.850700 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a7c46-3c7d-43e9-8d5c-f028a6e3b9de admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:25.850981 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a7c46-3c7d-43e9-8d5c-f028a6e3b9de admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:25.851288 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a7c46-3c7d-43e9-8d5c-f028a6e3b9de admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:25.852350 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2a7c46-3c7d-43e9-8d5c-f028a6e3b9de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:25.854317 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2a7c46-3c7d-43e9-8d5c-f028a6e3b9de admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:25.866486 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1286/2592] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:17:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:17:25.870460 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e66d26-8568-4ade-b86d-ea0316a471ac admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:25.881563 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e66d26-8568-4ade-b86d-ea0316a471ac admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:25.886395 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d37e904-0d0d-4f68-bab3-7928740456a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:25.887071 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d37e904-0d0d-4f68-bab3-7928740456a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:25.887389 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d37e904-0d0d-4f68-bab3-7928740456a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:25.887665 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d37e904-0d0d-4f68-bab3-7928740456a3 admin admin] PATH_INFO: `/v3/projects/07d589846f0e4149bf18aea4d0f2faa4/users/fadddcc4fd444e4aafccfc5a620ee1b3/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:25.896304 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:25.905219 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d37e904-0d0d-4f68-bab3-7928740456a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07d589846f0e4149bf18aea4d0f2faa4, user_id=fadddcc4fd444e4aafccfc5a620ee1b3, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:25.907980 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d37e904-0d0d-4f68-bab3-7928740456a3 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:25.937095 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1307/2593] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 15:17:25 2022] PUT /identity/v3/projects/07d589846f0e4149bf18aea4d0f2faa4/users/fadddcc4fd444e4aafccfc5a620ee1b3/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:17:25.947544 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051622ed-e534-4122-84d4-d0bc73c649f6 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:25.947839 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051622ed-e534-4122-84d4-d0bc73c649f6 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:25.948168 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051622ed-e534-4122-84d4-d0bc73c649f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:26.000237 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-051622ed-e534-4122-84d4-d0bc73c649f6 None None] MFA Rules not processed for user `fadddcc4fd444e4aafccfc5a620ee1b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:17:26.001737 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-051622ed-e534-4122-84d4-d0bc73c649f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:26.075513 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1287/2594] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:17:25 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:17:26.079616 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d37e904-0d0d-4f68-bab3-7928740456a3 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:26.145325 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d37e904-0d0d-4f68-bab3-7928740456a3 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:26.150966 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba277b-0e09-4df3-be8d-3cbe28c8bc0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:26.151612 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba277b-0e09-4df3-be8d-3cbe28c8bc0c admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:26.151893 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba277b-0e09-4df3-be8d-3cbe28c8bc0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:26.152164 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba277b-0e09-4df3-be8d-3cbe28c8bc0c admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:26.152810 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbba277b-0e09-4df3-be8d-3cbe28c8bc0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:26.155263 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbba277b-0e09-4df3-be8d-3cbe28c8bc0c admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:26.182229 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1308/2595] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 15:17:26 2022] POST /identity/v3/projects => generated 409 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:17:26.186620 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051622ed-e534-4122-84d4-d0bc73c649f6 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:26.197691 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051622ed-e534-4122-84d4-d0bc73c649f6 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:26.241723 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:26.259908 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5da418-460b-4f4f-9fbe-95cdb834de98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:26.260519 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5da418-460b-4f4f-9fbe-95cdb834de98 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:26.260827 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5da418-460b-4f4f-9fbe-95cdb834de98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:26.261102 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5da418-460b-4f4f-9fbe-95cdb834de98 admin admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:26.261736 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5da418-460b-4f4f-9fbe-95cdb834de98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:26.263907 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5da418-460b-4f4f-9fbe-95cdb834de98 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:26.289200 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1288/2596] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 15:17:26 2022] POST /identity/v3/users => generated 311 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:17:26.290926 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba277b-0e09-4df3-be8d-3cbe28c8bc0c admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:26.308783 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba277b-0e09-4df3-be8d-3cbe28c8bc0c admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:26.315666 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf87da7-c63a-4577-82c0-7851e2581b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:26.316364 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf87da7-c63a-4577-82c0-7851e2581b04 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:26.316701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf87da7-c63a-4577-82c0-7851e2581b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:26.316961 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf87da7-c63a-4577-82c0-7851e2581b04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:26.317669 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf87da7-c63a-4577-82c0-7851e2581b04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:26.320215 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf87da7-c63a-4577-82c0-7851e2581b04 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:26.333623 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1309/2597] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:17:26 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 15:17:26.339150 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5da418-460b-4f4f-9fbe-95cdb834de98 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:26.352025 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5da418-460b-4f4f-9fbe-95cdb834de98 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:26.358166 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf4e47-c67d-4166-b1f5-165805e2e988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:26.358821 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf4e47-c67d-4166-b1f5-165805e2e988 admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:26.359099 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf4e47-c67d-4166-b1f5-165805e2e988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:26.359423 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf4e47-c67d-4166-b1f5-165805e2e988 admin admin] PATH_INFO: `/v3/projects/0686a9923f734ed5b0e6f84acbf71e66/users/ed4496c23fb543f7a917e685b9eed646/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:26.374934 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbf4e47-c67d-4166-b1f5-165805e2e988 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0686a9923f734ed5b0e6f84acbf71e66, user_id=ed4496c23fb543f7a917e685b9eed646, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:26.377618 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbf4e47-c67d-4166-b1f5-165805e2e988 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:26.391709 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1289/2598] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 15:17:26 2022] PUT /identity/v3/projects/0686a9923f734ed5b0e6f84acbf71e66/users/ed4496c23fb543f7a917e685b9eed646/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:17:26.395433 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf87da7-c63a-4577-82c0-7851e2581b04 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:26.406533 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf87da7-c63a-4577-82c0-7851e2581b04 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:26.464790 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e346b2c1-c937-4bb0-90cf-94af7dbbe41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:26.465399 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346b2c1-c937-4bb0-90cf-94af7dbbe41a admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:26.465673 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346b2c1-c937-4bb0-90cf-94af7dbbe41a admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:26.465941 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346b2c1-c937-4bb0-90cf-94af7dbbe41a admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:26.467658 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e346b2c1-c937-4bb0-90cf-94af7dbbe41a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:26.469234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e346b2c1-c937-4bb0-90cf-94af7dbbe41a admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:26.483201 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1310/2599] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:17:26 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:17:26.487145 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf4e47-c67d-4166-b1f5-165805e2e988 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:26.498472 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf4e47-c67d-4166-b1f5-165805e2e988 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:26.503359 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b141b862-2076-48c3-933d-bdc6cddd360f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:26.504012 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141b862-2076-48c3-933d-bdc6cddd360f admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:26.504313 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141b862-2076-48c3-933d-bdc6cddd360f admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:26.504583 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141b862-2076-48c3-933d-bdc6cddd360f admin admin] PATH_INFO: `/v3/projects/0686a9923f734ed5b0e6f84acbf71e66/users/ed4496c23fb543f7a917e685b9eed646/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:26.506474 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b141b862-2076-48c3-933d-bdc6cddd360f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0686a9923f734ed5b0e6f84acbf71e66, user_id=ed4496c23fb543f7a917e685b9eed646, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:26.509129 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b141b862-2076-48c3-933d-bdc6cddd360f admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:26.520291 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1290/2600] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 15:17:26 2022] PUT /identity/v3/projects/0686a9923f734ed5b0e6f84acbf71e66/users/ed4496c23fb543f7a917e685b9eed646/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:17:26.532598 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb6816d-817d-4172-b614-da1bf559a491 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:26.532882 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb6816d-817d-4172-b614-da1bf559a491 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:26.533174 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb6816d-817d-4172-b614-da1bf559a491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:26.562020 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-1eb6816d-817d-4172-b614-da1bf559a491 None None] MFA Rules not processed for user `ed4496c23fb543f7a917e685b9eed646`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:17:26.563237 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-1eb6816d-817d-4172-b614-da1bf559a491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:26.608216 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:26.633792 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1311/2601] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:17:26 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:17:26.645280 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b141b862-2076-48c3-933d-bdc6cddd360f admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:26.647092 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-b141b862-2076-48c3-933d-bdc6cddd360f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:26.705497 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b141b862-2076-48c3-933d-bdc6cddd360f admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:26.709259 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3817551d-ebc0-4d5b-bf57-90f0083ad763 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:26.709852 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3817551d-ebc0-4d5b-bf57-90f0083ad763 service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:26.710132 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3817551d-ebc0-4d5b-bf57-90f0083ad763 service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:26.710405 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3817551d-ebc0-4d5b-bf57-90f0083ad763 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:26.719961 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3817551d-ebc0-4d5b-bf57-90f0083ad763 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:26.721382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3817551d-ebc0-4d5b-bf57-90f0083ad763 service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:26.730780 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1291/2602] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:17:26 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:17:26.898066 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb6816d-817d-4172-b614-da1bf559a491 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:26.908016 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb6816d-817d-4172-b614-da1bf559a491 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:26.915945 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe938ef-b000-4312-ab44-7d6237f7c7ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:26.916560 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe938ef-b000-4312-ab44-7d6237f7c7ee service cinder] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:26.916835 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe938ef-b000-4312-ab44-7d6237f7c7ee service cinder] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:26.917103 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe938ef-b000-4312-ab44-7d6237f7c7ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:26.975071 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe938ef-b000-4312-ab44-7d6237f7c7ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:26.976634 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe938ef-b000-4312-ab44-7d6237f7c7ee service cinder] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:26.995361 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1312/2603] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:17:26 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:17:30.853710 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf69dec-a0cc-42af-9dfe-06d85c8849ce None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:30.854203 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf69dec-a0cc-42af-9dfe-06d85c8849ce None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:30.854328 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf69dec-a0cc-42af-9dfe-06d85c8849ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:30.905231 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-fdf69dec-a0cc-42af-9dfe-06d85c8849ce None None] MFA Rules not processed for user `ecec9a5a18b24ae7ab055c37ea85ca18`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:17:30.906769 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-fdf69dec-a0cc-42af-9dfe-06d85c8849ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:30.971266 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1292/2604] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:17:30 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:17:31.275426 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe938ef-b000-4312-ab44-7d6237f7c7ee service cinder] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:31.276923 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-5fe938ef-b000-4312-ab44-7d6237f7c7ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:31.339541 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe938ef-b000-4312-ab44-7d6237f7c7ee service cinder] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:31.343949 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30480f7-9a24-4cc1-9977-7d0fb30cf63e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:31.344561 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30480f7-9a24-4cc1-9977-7d0fb30cf63e service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:31.344844 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30480f7-9a24-4cc1-9977-7d0fb30cf63e service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:31.345120 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30480f7-9a24-4cc1-9977-7d0fb30cf63e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:31.346937 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-b30480f7-9a24-4cc1-9977-7d0fb30cf63e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:31.356456 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30480f7-9a24-4cc1-9977-7d0fb30cf63e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:31.358508 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30480f7-9a24-4cc1-9977-7d0fb30cf63e service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:31.366452 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1313/2605] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:17:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:17:31.690374 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf69dec-a0cc-42af-9dfe-06d85c8849ce None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:31.692016 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-fdf69dec-a0cc-42af-9dfe-06d85c8849ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:31.702935 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:31.779571 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf69dec-a0cc-42af-9dfe-06d85c8849ce None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:31.785572 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cd9d82-ddc4-428e-8563-c4f40554c78e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcf302a170934c25b39699088dac326a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:31.786324 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd9d82-ddc4-428e-8563-c4f40554c78e service placement] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:31.786720 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd9d82-ddc4-428e-8563-c4f40554c78e service placement] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:31.787056 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd9d82-ddc4-428e-8563-c4f40554c78e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:31.788930 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-d4cd9d82-ddc4-428e-8563-c4f40554c78e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:31.803080 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cd9d82-ddc4-428e-8563-c4f40554c78e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:31.804838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cd9d82-ddc4-428e-8563-c4f40554c78e service placement] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:31.818255 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1293/2606] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:17:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:17:36.131417 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30480f7-9a24-4cc1-9977-7d0fb30cf63e service neutron] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:36.132963 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-b30480f7-9a24-4cc1-9977-7d0fb30cf63e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:36.143644 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30480f7-9a24-4cc1-9977-7d0fb30cf63e service neutron] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:36.147751 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192d7d74-bb41-4bed-abbe-3282fb3fbce9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:36.148539 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d7d74-bb41-4bed-abbe-3282fb3fbce9 service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:36.148851 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d7d74-bb41-4bed-abbe-3282fb3fbce9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:36.149201 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d7d74-bb41-4bed-abbe-3282fb3fbce9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:36.151160 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-192d7d74-bb41-4bed-abbe-3282fb3fbce9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:36.162745 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192d7d74-bb41-4bed-abbe-3282fb3fbce9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:36.164231 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192d7d74-bb41-4bed-abbe-3282fb3fbce9 service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:36.177005 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1314/2607] 10.222.0.30 () {60 vars in 1396 bytes} [Tue Aug 30 15:17:36 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3258 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:17:36.277765 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cd9d82-ddc4-428e-8563-c4f40554c78e service placement] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:36.287298 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cd9d82-ddc4-428e-8563-c4f40554c78e service placement] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:36.290935 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e40d384-0022-4185-8197-014f92942d0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:36.291571 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e40d384-0022-4185-8197-014f92942d0d service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:36.291853 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e40d384-0022-4185-8197-014f92942d0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:36.292129 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e40d384-0022-4185-8197-014f92942d0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:36.294621 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-0e40d384-0022-4185-8197-014f92942d0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:36.362779 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e40d384-0022-4185-8197-014f92942d0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:36.364065 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e40d384-0022-4185-8197-014f92942d0d service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:36.375335 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1294/2608] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:17:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:17:37.697112 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192d7d74-bb41-4bed-abbe-3282fb3fbce9 service neutron] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:37.708306 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192d7d74-bb41-4bed-abbe-3282fb3fbce9 service neutron] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:37.712828 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528a74b6-aff3-43e5-974f-0d719aa4743a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:37.713447 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a74b6-aff3-43e5-974f-0d719aa4743a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:37.713725 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a74b6-aff3-43e5-974f-0d719aa4743a admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:37.713996 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a74b6-aff3-43e5-974f-0d719aa4743a admin admin] PATH_INFO: `/v3/users/fadddcc4fd444e4aafccfc5a620ee1b3` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:37.715124 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528a74b6-aff3-43e5-974f-0d719aa4743a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fadddcc4fd444e4aafccfc5a620ee1b3)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:37.717238 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528a74b6-aff3-43e5-974f-0d719aa4743a admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:37.773473 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-528a74b6-aff3-43e5-974f-0d719aa4743a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fadddcc4fd444e4aafccfc5a620ee1b3'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:37.774181 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-528a74b6-aff3-43e5-974f-0d719aa4743a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fadddcc4fd444e4aafccfc5a620ee1b3'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:37.783099 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-528a74b6-aff3-43e5-974f-0d719aa4743a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fadddcc4fd444e4aafccfc5a620ee1b3'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:37.790666 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-528a74b6-aff3-43e5-974f-0d719aa4743a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fadddcc4fd444e4aafccfc5a620ee1b3'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:37.791425 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-528a74b6-aff3-43e5-974f-0d719aa4743a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fadddcc4fd444e4aafccfc5a620ee1b3'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:37.792851 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:37.814686 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1315/2609] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 15:17:37 2022] DELETE /identity/v3/users/fadddcc4fd444e4aafccfc5a620ee1b3 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:17:37.819369 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b7cfb-02e2-4213-ad03-602e4be52358 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:37.819900 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b7cfb-02e2-4213-ad03-602e4be52358 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:37.820009 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b7cfb-02e2-4213-ad03-602e4be52358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:37.843826 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-8e8b7cfb-02e2-4213-ad03-602e4be52358 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:17:37.844846 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-8e8b7cfb-02e2-4213-ad03-602e4be52358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:37.912017 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1295/2610] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:17:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:17:37.922329 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528a74b6-aff3-43e5-974f-0d719aa4743a admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:37.924014 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-528a74b6-aff3-43e5-974f-0d719aa4743a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:38.033319 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528a74b6-aff3-43e5-974f-0d719aa4743a admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:38.040992 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb878a0e-7086-4e13-a425-637039097f91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:38.042086 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb878a0e-7086-4e13-a425-637039097f91 service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:38.042595 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb878a0e-7086-4e13-a425-637039097f91 service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:38.042952 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb878a0e-7086-4e13-a425-637039097f91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:38.054675 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb878a0e-7086-4e13-a425-637039097f91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:38.056419 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb878a0e-7086-4e13-a425-637039097f91 service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:38.075866 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1316/2611] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:17:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:17:38.393359 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8b7cfb-02e2-4213-ad03-602e4be52358 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:38.395782 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-8e8b7cfb-02e2-4213-ad03-602e4be52358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:38.409840 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8b7cfb-02e2-4213-ad03-602e4be52358 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:38.416005 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd9353a-116b-4ae5-a6ed-c321eb6883fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:38.418623 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9353a-116b-4ae5-a6ed-c321eb6883fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:38.418623 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9353a-116b-4ae5-a6ed-c321eb6883fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:38.418623 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9353a-116b-4ae5-a6ed-c321eb6883fb admin admin] PATH_INFO: `/v3/projects/07d589846f0e4149bf18aea4d0f2faa4` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:38.419281 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd9353a-116b-4ae5-a6ed-c321eb6883fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=07d589846f0e4149bf18aea4d0f2faa4)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:38.421418 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd9353a-116b-4ae5-a6ed-c321eb6883fb admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:38.467821 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:38.486944 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-1cd9353a-116b-4ae5-a6ed-c321eb6883fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07d589846f0e4149bf18aea4d0f2faa4', 'request_id': 'req-1cd9353a-116b-4ae5-a6ed-c321eb6883fb'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:38.493704 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-1cd9353a-116b-4ae5-a6ed-c321eb6883fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07d589846f0e4149bf18aea4d0f2faa4', 'request_id': 'req-1cd9353a-116b-4ae5-a6ed-c321eb6883fb'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:38.501679 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-1cd9353a-116b-4ae5-a6ed-c321eb6883fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07d589846f0e4149bf18aea4d0f2faa4', 'request_id': 'req-1cd9353a-116b-4ae5-a6ed-c321eb6883fb'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:38.519569 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-1cd9353a-116b-4ae5-a6ed-c321eb6883fb admin admin] The token cache is being invalidate because project 07d589846f0e4149bf18aea4d0f2faa4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:17:38.519868 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-1cd9353a-116b-4ae5-a6ed-c321eb6883fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:38.520527 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-1cd9353a-116b-4ae5-a6ed-c321eb6883fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:38.524880 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1296/2612] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 15:17:38 2022] DELETE /identity/v3/projects/07d589846f0e4149bf18aea4d0f2faa4 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:17:38.526047 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb878a0e-7086-4e13-a425-637039097f91 service neutron] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:38.538690 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb878a0e-7086-4e13-a425-637039097f91 service neutron] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:38.612069 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f82b9bc-60e2-4808-8919-83fcb8699e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:38.613121 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f82b9bc-60e2-4808-8919-83fcb8699e3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:38.613609 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f82b9bc-60e2-4808-8919-83fcb8699e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:38.614091 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f82b9bc-60e2-4808-8919-83fcb8699e3b admin admin] PATH_INFO: `/v3/users/ed4496c23fb543f7a917e685b9eed646` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:38.616048 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f82b9bc-60e2-4808-8919-83fcb8699e3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed4496c23fb543f7a917e685b9eed646)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:38.619880 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f82b9bc-60e2-4808-8919-83fcb8699e3b admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:38.689088 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-9f82b9bc-60e2-4808-8919-83fcb8699e3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed4496c23fb543f7a917e685b9eed646'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:38.689735 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-9f82b9bc-60e2-4808-8919-83fcb8699e3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed4496c23fb543f7a917e685b9eed646'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:38.697624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-9f82b9bc-60e2-4808-8919-83fcb8699e3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed4496c23fb543f7a917e685b9eed646'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:38.705815 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-9f82b9bc-60e2-4808-8919-83fcb8699e3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed4496c23fb543f7a917e685b9eed646'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:38.706499 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-9f82b9bc-60e2-4808-8919-83fcb8699e3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed4496c23fb543f7a917e685b9eed646'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:38.724430 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1317/2613] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 15:17:38 2022] DELETE /identity/v3/users/ed4496c23fb543f7a917e685b9eed646 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:17:39.246047 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd9353a-116b-4ae5-a6ed-c321eb6883fb admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:39.248483 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-1cd9353a-116b-4ae5-a6ed-c321eb6883fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:39.286945 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd9353a-116b-4ae5-a6ed-c321eb6883fb admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:39.369710 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d72f3-76d7-4e33-a08a-79888f73f0b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:39.370361 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d72f3-76d7-4e33-a08a-79888f73f0b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:39.370650 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d72f3-76d7-4e33-a08a-79888f73f0b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:39.371083 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d72f3-76d7-4e33-a08a-79888f73f0b8 admin admin] PATH_INFO: `/v3/projects/0686a9923f734ed5b0e6f84acbf71e66` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:39.372379 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583d72f3-76d7-4e33-a08a-79888f73f0b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0686a9923f734ed5b0e6f84acbf71e66)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:39.374544 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583d72f3-76d7-4e33-a08a-79888f73f0b8 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:39.447088 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-583d72f3-76d7-4e33-a08a-79888f73f0b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0686a9923f734ed5b0e6f84acbf71e66', 'request_id': 'req-583d72f3-76d7-4e33-a08a-79888f73f0b8'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:39.456210 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-583d72f3-76d7-4e33-a08a-79888f73f0b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0686a9923f734ed5b0e6f84acbf71e66', 'request_id': 'req-583d72f3-76d7-4e33-a08a-79888f73f0b8'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:39.462874 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-583d72f3-76d7-4e33-a08a-79888f73f0b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0686a9923f734ed5b0e6f84acbf71e66', 'request_id': 'req-583d72f3-76d7-4e33-a08a-79888f73f0b8'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:39.477970 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-583d72f3-76d7-4e33-a08a-79888f73f0b8 admin admin] The token cache is being invalidate because project 0686a9923f734ed5b0e6f84acbf71e66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:17:39.478233 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-583d72f3-76d7-4e33-a08a-79888f73f0b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:39.478884 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-583d72f3-76d7-4e33-a08a-79888f73f0b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:39.480759 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1297/2614] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 15:17:39 2022] DELETE /identity/v3/projects/0686a9923f734ed5b0e6f84acbf71e66 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:17:39.485948 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a655650c-413f-4c9c-8810-e9d5806c681b None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:39.486363 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a655650c-413f-4c9c-8810-e9d5806c681b None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:39.486614 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a655650c-413f-4c9c-8810-e9d5806c681b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:39.496322 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:39.510146 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-a655650c-413f-4c9c-8810-e9d5806c681b None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:17:39.512101 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-a655650c-413f-4c9c-8810-e9d5806c681b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:39.583636 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1318/2615] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:17:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:17:39.590918 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2bf7dd-5fd3-44a2-b623-59880264dd8d None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:39.591200 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2bf7dd-5fd3-44a2-b623-59880264dd8d None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:39.591491 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2bf7dd-5fd3-44a2-b623-59880264dd8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:39.623582 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-0f2bf7dd-5fd3-44a2-b623-59880264dd8d None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:17:39.624926 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-0f2bf7dd-5fd3-44a2-b623-59880264dd8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:39.643860 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a655650c-413f-4c9c-8810-e9d5806c681b None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:39.650549 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1298/2616] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:17:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:17:39.659676 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a655650c-413f-4c9c-8810-e9d5806c681b None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:39.666347 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a28a72-54e6-4797-ae20-80275d9625b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:39.667102 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a28a72-54e6-4797-ae20-80275d9625b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:39.667512 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a28a72-54e6-4797-ae20-80275d9625b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:39.667884 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a28a72-54e6-4797-ae20-80275d9625b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:39.668643 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a28a72-54e6-4797-ae20-80275d9625b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:17:39.669218 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a28a72-54e6-4797-ae20-80275d9625b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:39.671619 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a28a72-54e6-4797-ae20-80275d9625b3 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:39.692733 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1319/2617] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 15:17:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:17:39.694039 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2bf7dd-5fd3-44a2-b623-59880264dd8d None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:39.704366 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2bf7dd-5fd3-44a2-b623-59880264dd8d None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:39.709698 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fea2ef-b573-4833-b76b-b42c3192954d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:39.710438 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fea2ef-b573-4833-b76b-b42c3192954d admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:39.710818 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fea2ef-b573-4833-b76b-b42c3192954d admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:39.711198 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fea2ef-b573-4833-b76b-b42c3192954d admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:39.711972 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fea2ef-b573-4833-b76b-b42c3192954d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:39.714000 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fea2ef-b573-4833-b76b-b42c3192954d admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:39.751628 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1299/2618] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 15:17:39 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:17:39.753574 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a28a72-54e6-4797-ae20-80275d9625b3 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:39.767177 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a28a72-54e6-4797-ae20-80275d9625b3 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:39.869881 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2625eb58-19f7-4f18-a6c7-827073b865b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:39.870474 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2625eb58-19f7-4f18-a6c7-827073b865b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:39.870766 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2625eb58-19f7-4f18-a6c7-827073b865b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:39.871010 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2625eb58-19f7-4f18-a6c7-827073b865b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:39.871894 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2625eb58-19f7-4f18-a6c7-827073b865b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:39.874400 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2625eb58-19f7-4f18-a6c7-827073b865b6 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:39.914817 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1320/2619] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 15:17:39 2022] POST /identity/v3/users => generated 286 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:17:39.919584 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fea2ef-b573-4833-b76b-b42c3192954d admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:39.934100 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fea2ef-b573-4833-b76b-b42c3192954d admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:39.939225 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b56cc8e-692d-4de8-87ce-c7e7fa1cbfa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:39.939851 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56cc8e-692d-4de8-87ce-c7e7fa1cbfa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:39.940135 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56cc8e-692d-4de8-87ce-c7e7fa1cbfa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:39.940412 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56cc8e-692d-4de8-87ce-c7e7fa1cbfa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:39.941057 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b56cc8e-692d-4de8-87ce-c7e7fa1cbfa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:39.942679 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b56cc8e-692d-4de8-87ce-c7e7fa1cbfa1 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:39.953875 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1300/2620] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:17:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:17:39.960287 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2625eb58-19f7-4f18-a6c7-827073b865b6 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:39.972602 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2625eb58-19f7-4f18-a6c7-827073b865b6 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:39.977480 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d157bdb-29af-40ee-b7a9-d24895d09628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:39.978152 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d157bdb-29af-40ee-b7a9-d24895d09628 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:39.978497 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d157bdb-29af-40ee-b7a9-d24895d09628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:39.978836 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d157bdb-29af-40ee-b7a9-d24895d09628 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:39.979607 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d157bdb-29af-40ee-b7a9-d24895d09628 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:39.981246 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d157bdb-29af-40ee-b7a9-d24895d09628 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:39.992732 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1321/2621] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:17:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:17:39.997196 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b56cc8e-692d-4de8-87ce-c7e7fa1cbfa1 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:40.004572 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:40.009216 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b56cc8e-692d-4de8-87ce-c7e7fa1cbfa1 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:40.017832 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194c587f-e2b0-42c9-a795-0dfc6e7e5bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:40.018861 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c587f-e2b0-42c9-a795-0dfc6e7e5bc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:40.019406 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c587f-e2b0-42c9-a795-0dfc6e7e5bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:40.019942 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c587f-e2b0-42c9-a795-0dfc6e7e5bc8 admin admin] PATH_INFO: `/v3/projects/8f8f2793098f4e74b8438921c16c9614/users/f5c0f1ff45de40c3ba5bdf51cb8e7522/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:40.036964 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194c587f-e2b0-42c9-a795-0dfc6e7e5bc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f8f2793098f4e74b8438921c16c9614, user_id=f5c0f1ff45de40c3ba5bdf51cb8e7522, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:40.039834 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194c587f-e2b0-42c9-a795-0dfc6e7e5bc8 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:40.053661 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1301/2622] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 15:17:39 2022] PUT /identity/v3/projects/8f8f2793098f4e74b8438921c16c9614/users/f5c0f1ff45de40c3ba5bdf51cb8e7522/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:17:40.065094 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c1567-2d38-42fa-9531-4049cd5d57b8 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:40.065377 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c1567-2d38-42fa-9531-4049cd5d57b8 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:40.065773 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c1567-2d38-42fa-9531-4049cd5d57b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:40.096997 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-c49c1567-2d38-42fa-9531-4049cd5d57b8 None None] MFA Rules not processed for user `f5c0f1ff45de40c3ba5bdf51cb8e7522`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:17:40.098213 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-c49c1567-2d38-42fa-9531-4049cd5d57b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:40.161447 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:40.169097 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1322/2623] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:17:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:17:40.182721 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194c587f-e2b0-42c9-a795-0dfc6e7e5bc8 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:40.184990 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-194c587f-e2b0-42c9-a795-0dfc6e7e5bc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:40.279588 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194c587f-e2b0-42c9-a795-0dfc6e7e5bc8 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:40.284016 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e859950f-8ef4-4102-b38c-11decba293a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:40.284757 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859950f-8ef4-4102-b38c-11decba293a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:40.285490 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859950f-8ef4-4102-b38c-11decba293a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:40.285490 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859950f-8ef4-4102-b38c-11decba293a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:40.295418 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e859950f-8ef4-4102-b38c-11decba293a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:40.297142 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e859950f-8ef4-4102-b38c-11decba293a5 service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:40.309018 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1302/2624] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:17:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:17:40.332907 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49c1567-2d38-42fa-9531-4049cd5d57b8 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:40.409380 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49c1567-2d38-42fa-9531-4049cd5d57b8 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:40.417099 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2614252d-2171-435d-bf0c-e1ffa741197f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:40.417753 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614252d-2171-435d-bf0c-e1ffa741197f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:40.418035 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614252d-2171-435d-bf0c-e1ffa741197f admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:40.418312 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614252d-2171-435d-bf0c-e1ffa741197f admin admin] PATH_INFO: `/v3/users/f5c0f1ff45de40c3ba5bdf51cb8e7522` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:40.420457 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2614252d-2171-435d-bf0c-e1ffa741197f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5c0f1ff45de40c3ba5bdf51cb8e7522)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:40.423099 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2614252d-2171-435d-bf0c-e1ffa741197f admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:40.489401 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-2614252d-2171-435d-bf0c-e1ffa741197f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5c0f1ff45de40c3ba5bdf51cb8e7522'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:40.490051 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-2614252d-2171-435d-bf0c-e1ffa741197f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5c0f1ff45de40c3ba5bdf51cb8e7522'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:40.497881 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-2614252d-2171-435d-bf0c-e1ffa741197f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5c0f1ff45de40c3ba5bdf51cb8e7522'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:40.505512 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-2614252d-2171-435d-bf0c-e1ffa741197f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5c0f1ff45de40c3ba5bdf51cb8e7522'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:40.506293 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-2614252d-2171-435d-bf0c-e1ffa741197f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5c0f1ff45de40c3ba5bdf51cb8e7522'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:40.524379 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1323/2625] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 15:17:40 2022] DELETE /identity/v3/users/f5c0f1ff45de40c3ba5bdf51cb8e7522 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:17:40.529935 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd4dd56-2563-4788-8c89-612a6e4e9fb9 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:40.530407 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd4dd56-2563-4788-8c89-612a6e4e9fb9 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:40.531378 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd4dd56-2563-4788-8c89-612a6e4e9fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:40.551500 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-acd4dd56-2563-4788-8c89-612a6e4e9fb9 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:17:40.552720 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-acd4dd56-2563-4788-8c89-612a6e4e9fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:40.615118 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1303/2626] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:17:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:17:40.625655 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2614252d-2171-435d-bf0c-e1ffa741197f admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:40.627286 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-2614252d-2171-435d-bf0c-e1ffa741197f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:40.705496 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2614252d-2171-435d-bf0c-e1ffa741197f admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:40.710155 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f5ec45-4f1b-4cb7-9451-2515a28ba2b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:40.710832 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5ec45-4f1b-4cb7-9451-2515a28ba2b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:40.711281 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5ec45-4f1b-4cb7-9451-2515a28ba2b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:40.711501 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5ec45-4f1b-4cb7-9451-2515a28ba2b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:40.724376 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f5ec45-4f1b-4cb7-9451-2515a28ba2b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:40.725817 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f5ec45-4f1b-4cb7-9451-2515a28ba2b9 service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:40.737916 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1324/2627] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:17:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:17:41.317654 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd4dd56-2563-4788-8c89-612a6e4e9fb9 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:41.319924 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-acd4dd56-2563-4788-8c89-612a6e4e9fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:41.346908 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd4dd56-2563-4788-8c89-612a6e4e9fb9 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:41.353271 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca6d2d-c3a9-4108-9ae1-2f37bee48d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:41.353916 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca6d2d-c3a9-4108-9ae1-2f37bee48d39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:41.354186 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca6d2d-c3a9-4108-9ae1-2f37bee48d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:41.355052 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca6d2d-c3a9-4108-9ae1-2f37bee48d39 admin admin] PATH_INFO: `/v3/projects/8f8f2793098f4e74b8438921c16c9614` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:41.364346 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ca6d2d-c3a9-4108-9ae1-2f37bee48d39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f8f2793098f4e74b8438921c16c9614)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:41.371304 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ca6d2d-c3a9-4108-9ae1-2f37bee48d39 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:41.444249 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-56ca6d2d-c3a9-4108-9ae1-2f37bee48d39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f8f2793098f4e74b8438921c16c9614', 'request_id': 'req-56ca6d2d-c3a9-4108-9ae1-2f37bee48d39'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:41.455602 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-56ca6d2d-c3a9-4108-9ae1-2f37bee48d39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f8f2793098f4e74b8438921c16c9614', 'request_id': 'req-56ca6d2d-c3a9-4108-9ae1-2f37bee48d39'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:41.462674 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-56ca6d2d-c3a9-4108-9ae1-2f37bee48d39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f8f2793098f4e74b8438921c16c9614', 'request_id': 'req-56ca6d2d-c3a9-4108-9ae1-2f37bee48d39'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:41.483930 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-56ca6d2d-c3a9-4108-9ae1-2f37bee48d39 admin admin] The token cache is being invalidate because project 8f8f2793098f4e74b8438921c16c9614 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:17:41.484323 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-56ca6d2d-c3a9-4108-9ae1-2f37bee48d39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:41.485109 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-56ca6d2d-c3a9-4108-9ae1-2f37bee48d39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:41.491903 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1304/2628] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 15:17:41 2022] DELETE /identity/v3/projects/8f8f2793098f4e74b8438921c16c9614 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:17:41.492845 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0ba8b-5f5c-469d-9429-67ecb8bfc548 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:41.493345 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0ba8b-5f5c-469d-9429-67ecb8bfc548 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:41.493466 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0ba8b-5f5c-469d-9429-67ecb8bfc548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:41.542985 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-02f0ba8b-5f5c-469d-9429-67ecb8bfc548 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:17:41.545003 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-02f0ba8b-5f5c-469d-9429-67ecb8bfc548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:41.550087 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca6d2d-c3a9-4108-9ae1-2f37bee48d39 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:41.551759 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-56ca6d2d-c3a9-4108-9ae1-2f37bee48d39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:41.571317 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:41.585141 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:41.671472 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1325/2629] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:17:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 15:17:41.672288 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f0ba8b-5f5c-469d-9429-67ecb8bfc548 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:41.673825 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-02f0ba8b-5f5c-469d-9429-67ecb8bfc548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:41.688936 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f0ba8b-5f5c-469d-9429-67ecb8bfc548 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:41.695155 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7142c9d-ff42-4b5f-90e2-594f2acbbd8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:41.695885 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7142c9d-ff42-4b5f-90e2-594f2acbbd8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:41.696250 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7142c9d-ff42-4b5f-90e2-594f2acbbd8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:41.696431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7142c9d-ff42-4b5f-90e2-594f2acbbd8d admin admin] PATH_INFO: `/v3/users/35f28006916040c3b988de6c15093166` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:41.698859 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7142c9d-ff42-4b5f-90e2-594f2acbbd8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=35f28006916040c3b988de6c15093166)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:41.704474 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7142c9d-ff42-4b5f-90e2-594f2acbbd8d admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:41.705270 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca6d2d-c3a9-4108-9ae1-2f37bee48d39 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:41.711621 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a1ea1-22e1-463b-8ceb-418f9b0116bf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcf302a170934c25b39699088dac326a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:41.712311 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a1ea1-22e1-463b-8ceb-418f9b0116bf service placement] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:41.712643 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a1ea1-22e1-463b-8ceb-418f9b0116bf service placement] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:41.712985 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a1ea1-22e1-463b-8ceb-418f9b0116bf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:41.714846 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-db7a1ea1-22e1-463b-8ceb-418f9b0116bf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:41.728405 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7a1ea1-22e1-463b-8ceb-418f9b0116bf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:41.730113 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7a1ea1-22e1-463b-8ceb-418f9b0116bf service placement] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:41.742057 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1305/2630] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:17:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:17:41.743249 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b98262f-6d05-4fae-be4e-15efcdaea835 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:41.743607 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b98262f-6d05-4fae-be4e-15efcdaea835 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:41.744083 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b98262f-6d05-4fae-be4e-15efcdaea835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:41.770789 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-1b98262f-6d05-4fae-be4e-15efcdaea835 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:17:41.772454 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-1b98262f-6d05-4fae-be4e-15efcdaea835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:41.784844 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1306/2631] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:17:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:17:41.789006 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b98262f-6d05-4fae-be4e-15efcdaea835 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:41.810751 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b98262f-6d05-4fae-be4e-15efcdaea835 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:41.816394 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c64bb7-9db7-4452-81ec-1959206d6492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:41.817038 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c64bb7-9db7-4452-81ec-1959206d6492 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:41.817442 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c64bb7-9db7-4452-81ec-1959206d6492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:41.817840 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c64bb7-9db7-4452-81ec-1959206d6492 admin admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:41.818530 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c64bb7-9db7-4452-81ec-1959206d6492 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:17:41.818834 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c64bb7-9db7-4452-81ec-1959206d6492 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:41.820652 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c64bb7-9db7-4452-81ec-1959206d6492 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:41.836540 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1307/2632] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 15:17:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:17:41.841964 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c64bb7-9db7-4452-81ec-1959206d6492 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:41.853356 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c64bb7-9db7-4452-81ec-1959206d6492 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:41.868721 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ba1a1c-cc8b-4629-8c2f-453162565af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:41.869380 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ba1a1c-cc8b-4629-8c2f-453162565af3 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:41.869660 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ba1a1c-cc8b-4629-8c2f-453162565af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:41.869937 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ba1a1c-cc8b-4629-8c2f-453162565af3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:41.870821 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ba1a1c-cc8b-4629-8c2f-453162565af3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:41.872829 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ba1a1c-cc8b-4629-8c2f-453162565af3 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:41.914412 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-a7142c9d-ff42-4b5f-90e2-594f2acbbd8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35f28006916040c3b988de6c15093166'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:41.918094 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-a7142c9d-ff42-4b5f-90e2-594f2acbbd8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35f28006916040c3b988de6c15093166'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:41.918248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1308/2633] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 15:17:41 2022] POST /identity/v3/projects => generated 373 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:17:41.920405 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ba1a1c-cc8b-4629-8c2f-453162565af3 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:41.924268 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-84ba1a1c-cc8b-4629-8c2f-453162565af3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:41.928292 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-a7142c9d-ff42-4b5f-90e2-594f2acbbd8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35f28006916040c3b988de6c15093166'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:41.941876 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-a7142c9d-ff42-4b5f-90e2-594f2acbbd8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35f28006916040c3b988de6c15093166'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:41.942743 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-a7142c9d-ff42-4b5f-90e2-594f2acbbd8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35f28006916040c3b988de6c15093166'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:41.971762 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1326/2634] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 15:17:41 2022] DELETE /identity/v3/users/35f28006916040c3b988de6c15093166 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:17:42.012788 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ba1a1c-cc8b-4629-8c2f-453162565af3 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:42.091830 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96bdd4-a494-462a-840d-c6765de39f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:42.092737 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96bdd4-a494-462a-840d-c6765de39f29 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:42.093241 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96bdd4-a494-462a-840d-c6765de39f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:42.093678 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96bdd4-a494-462a-840d-c6765de39f29 admin admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:42.094510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea96bdd4-a494-462a-840d-c6765de39f29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:42.096788 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea96bdd4-a494-462a-840d-c6765de39f29 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:42.113384 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:42.129250 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1309/2635] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 15:17:41 2022] POST /identity/v3/users => generated 287 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:17:42.133442 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7142c9d-ff42-4b5f-90e2-594f2acbbd8d admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:42.145103 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7142c9d-ff42-4b5f-90e2-594f2acbbd8d admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:42.152153 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ffd166-ff5f-4b35-9563-b704820938cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:42.152503 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ffd166-ff5f-4b35-9563-b704820938cf admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:42.153072 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ffd166-ff5f-4b35-9563-b704820938cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:42.154053 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ffd166-ff5f-4b35-9563-b704820938cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:42.155082 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ffd166-ff5f-4b35-9563-b704820938cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:42.156839 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ffd166-ff5f-4b35-9563-b704820938cf admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:42.171967 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1327/2636] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:17:42 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 15:17:42.184491 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96bdd4-a494-462a-840d-c6765de39f29 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:42.192378 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ffd166-ff5f-4b35-9563-b704820938cf admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:42.195291 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-83ffd166-ff5f-4b35-9563-b704820938cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:42.204488 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96bdd4-a494-462a-840d-c6765de39f29 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:42.210963 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ffd166-ff5f-4b35-9563-b704820938cf admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:42.218240 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa86f28-b0f3-4d8b-9a43-6fef2377f6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:42.223825 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa86f28-b0f3-4d8b-9a43-6fef2377f6c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:42.223825 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa86f28-b0f3-4d8b-9a43-6fef2377f6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:42.223825 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa86f28-b0f3-4d8b-9a43-6fef2377f6c6 admin admin] PATH_INFO: `/v3/projects/65bab38256b44ad6b4b573a2441eb6d3` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:42.223825 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa86f28-b0f3-4d8b-9a43-6fef2377f6c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65bab38256b44ad6b4b573a2441eb6d3)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:42.223825 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa86f28-b0f3-4d8b-9a43-6fef2377f6c6 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:42.234372 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fc25c1-c636-4e9b-a623-9d368d60a5a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:42.238133 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc25c1-c636-4e9b-a623-9d368d60a5a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:42.238423 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc25c1-c636-4e9b-a623-9d368d60a5a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:42.238704 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc25c1-c636-4e9b-a623-9d368d60a5a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:42.239401 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fc25c1-c636-4e9b-a623-9d368d60a5a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:42.241179 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fc25c1-c636-4e9b-a623-9d368d60a5a3 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:42.255260 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1310/2637] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:17:42 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:17:42.257365 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fc25c1-c636-4e9b-a623-9d368d60a5a3 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:42.269467 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fc25c1-c636-4e9b-a623-9d368d60a5a3 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:42.282093 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-9fa86f28-b0f3-4d8b-9a43-6fef2377f6c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65bab38256b44ad6b4b573a2441eb6d3', 'request_id': 'req-9fa86f28-b0f3-4d8b-9a43-6fef2377f6c6'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:42.290395 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-9fa86f28-b0f3-4d8b-9a43-6fef2377f6c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65bab38256b44ad6b4b573a2441eb6d3', 'request_id': 'req-9fa86f28-b0f3-4d8b-9a43-6fef2377f6c6'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:42.308270 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-9fa86f28-b0f3-4d8b-9a43-6fef2377f6c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65bab38256b44ad6b4b573a2441eb6d3', 'request_id': 'req-9fa86f28-b0f3-4d8b-9a43-6fef2377f6c6'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:42.323521 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-9fa86f28-b0f3-4d8b-9a43-6fef2377f6c6 admin admin] The token cache is being invalidate because project 65bab38256b44ad6b4b573a2441eb6d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:17:42.323802 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-9fa86f28-b0f3-4d8b-9a43-6fef2377f6c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:42.324718 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-9fa86f28-b0f3-4d8b-9a43-6fef2377f6c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:42.327266 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1328/2638] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 15:17:42 2022] DELETE /identity/v3/projects/65bab38256b44ad6b4b573a2441eb6d3 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:17:42.332977 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be463b5-c581-45bf-9b6b-88f5b4f98208 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:42.333258 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be463b5-c581-45bf-9b6b-88f5b4f98208 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:42.333526 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be463b5-c581-45bf-9b6b-88f5b4f98208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:42.342704 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f4e2cd-275b-4a2d-bfae-144e33f35d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:42.343442 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4e2cd-275b-4a2d-bfae-144e33f35d2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:42.343851 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4e2cd-275b-4a2d-bfae-144e33f35d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:42.344127 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:42.344273 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4e2cd-275b-4a2d-bfae-144e33f35d2b admin admin] PATH_INFO: `/v3/projects/9260c0f22e10463294d486f5a442a3bc/users/0df8887828d541a2b5df73a28ee717a0/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:42.359263 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-8be463b5-c581-45bf-9b6b-88f5b4f98208 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:17:42.359678 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-8be463b5-c581-45bf-9b6b-88f5b4f98208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:42.362090 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f4e2cd-275b-4a2d-bfae-144e33f35d2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9260c0f22e10463294d486f5a442a3bc, user_id=0df8887828d541a2b5df73a28ee717a0, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:42.365170 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f4e2cd-275b-4a2d-bfae-144e33f35d2b admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:42.368655 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1329/2639] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:17:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 15:17:42.377651 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be7e6b-ef45-4931-aa80-f2d2fb1e0dbf None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:42.377972 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be7e6b-ef45-4931-aa80-f2d2fb1e0dbf None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:42.378197 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be7e6b-ef45-4931-aa80-f2d2fb1e0dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:42.380806 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1311/2640] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 15:17:42 2022] PUT /identity/v3/projects/9260c0f22e10463294d486f5a442a3bc/users/0df8887828d541a2b5df73a28ee717a0/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:17:42.391490 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250dd823-a706-4aca-a960-ac8fce23e903 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:42.391799 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250dd823-a706-4aca-a960-ac8fce23e903 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:42.392028 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250dd823-a706-4aca-a960-ac8fce23e903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:42.407307 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-21be7e6b-ef45-4931-aa80-f2d2fb1e0dbf None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:17:42.408508 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-21be7e6b-ef45-4931-aa80-f2d2fb1e0dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:42.425691 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-250dd823-a706-4aca-a960-ac8fce23e903 None None] MFA Rules not processed for user `0df8887828d541a2b5df73a28ee717a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:17:42.428081 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-250dd823-a706-4aca-a960-ac8fce23e903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:42.471043 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1330/2641] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:17:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 15:17:42.475153 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be7e6b-ef45-4931-aa80-f2d2fb1e0dbf None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:42.484447 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be7e6b-ef45-4931-aa80-f2d2fb1e0dbf None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:42.489294 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce1f851-fac4-47bf-a082-10b1e487009c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:42.489891 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1f851-fac4-47bf-a082-10b1e487009c admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:42.490164 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1f851-fac4-47bf-a082-10b1e487009c admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:42.490430 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1f851-fac4-47bf-a082-10b1e487009c admin admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:42.491071 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce1f851-fac4-47bf-a082-10b1e487009c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:17:42.491396 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce1f851-fac4-47bf-a082-10b1e487009c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:42.492978 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce1f851-fac4-47bf-a082-10b1e487009c admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:42.505707 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1331/2642] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 15:17:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:17:42.507854 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1312/2643] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:17:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 15:17:42.512200 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce1f851-fac4-47bf-a082-10b1e487009c admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:42.519907 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250dd823-a706-4aca-a960-ac8fce23e903 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:42.521421 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-250dd823-a706-4aca-a960-ac8fce23e903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:42.522612 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce1f851-fac4-47bf-a082-10b1e487009c admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:42.527535 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25239386-ada3-428a-8ef0-f101556d2d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:42.528151 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25239386-ada3-428a-8ef0-f101556d2d8b admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:42.528447 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25239386-ada3-428a-8ef0-f101556d2d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:42.528713 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25239386-ada3-428a-8ef0-f101556d2d8b admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:42.529342 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25239386-ada3-428a-8ef0-f101556d2d8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:42.531200 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25239386-ada3-428a-8ef0-f101556d2d8b admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:42.559066 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1332/2644] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 15:17:42 2022] POST /identity/v3/projects => generated 405 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:17:42.564141 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25239386-ada3-428a-8ef0-f101556d2d8b admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:42.589164 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250dd823-a706-4aca-a960-ac8fce23e903 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:42.593838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:42.627767 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25239386-ada3-428a-8ef0-f101556d2d8b admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:42.632743 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d0783e-9715-426d-84d7-98290d2e1a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:42.633779 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d0783e-9715-426d-84d7-98290d2e1a7d admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:42.634060 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d0783e-9715-426d-84d7-98290d2e1a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:42.634344 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d0783e-9715-426d-84d7-98290d2e1a7d admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:42.634978 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d0783e-9715-426d-84d7-98290d2e1a7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:42.637105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d0783e-9715-426d-84d7-98290d2e1a7d admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:42.642215 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862d84a2-9e74-4330-8b58-a82609ea7ea1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:42.642818 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d84a2-9e74-4330-8b58-a82609ea7ea1 service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:42.643093 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d84a2-9e74-4330-8b58-a82609ea7ea1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:42.643392 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d84a2-9e74-4330-8b58-a82609ea7ea1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:42.662851 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1333/2645] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 15:17:42 2022] POST /identity/v3/users => generated 303 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:17:42.666767 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d0783e-9715-426d-84d7-98290d2e1a7d admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:42.676704 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d0783e-9715-426d-84d7-98290d2e1a7d admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:42.681823 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec71e4c6-91e0-48df-a65b-dc3e43aa826d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:42.682432 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71e4c6-91e0-48df-a65b-dc3e43aa826d admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:42.682714 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71e4c6-91e0-48df-a65b-dc3e43aa826d admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:42.682990 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71e4c6-91e0-48df-a65b-dc3e43aa826d admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:42.683663 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec71e4c6-91e0-48df-a65b-dc3e43aa826d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:42.685973 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec71e4c6-91e0-48df-a65b-dc3e43aa826d admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:42.696993 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1334/2646] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:17:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 15:17:42.701647 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec71e4c6-91e0-48df-a65b-dc3e43aa826d admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:42.708066 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862d84a2-9e74-4330-8b58-a82609ea7ea1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:42.709608 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862d84a2-9e74-4330-8b58-a82609ea7ea1 service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:42.712282 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec71e4c6-91e0-48df-a65b-dc3e43aa826d admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:42.718047 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b0770a-557f-46fd-b856-54ed81a3e189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:42.718649 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b0770a-557f-46fd-b856-54ed81a3e189 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:42.718926 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b0770a-557f-46fd-b856-54ed81a3e189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:42.719193 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b0770a-557f-46fd-b856-54ed81a3e189 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:42.719856 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b0770a-557f-46fd-b856-54ed81a3e189 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:42.720399 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1313/2647] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:17:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 15:17:42.721618 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b0770a-557f-46fd-b856-54ed81a3e189 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:42.735915 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1335/2648] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:17:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:17:42.739774 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862d84a2-9e74-4330-8b58-a82609ea7ea1 service cinder] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:42.753799 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862d84a2-9e74-4330-8b58-a82609ea7ea1 service cinder] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:42.759917 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb3d0e4-17d5-401e-b1aa-abe219260dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:42.760576 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb3d0e4-17d5-401e-b1aa-abe219260dbc admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:42.760856 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb3d0e4-17d5-401e-b1aa-abe219260dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:42.761130 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb3d0e4-17d5-401e-b1aa-abe219260dbc admin admin] PATH_INFO: `/v3/projects/40636f0705234176be95cae0e97b604c/users/25016ba96e2144329ef5b62449974619/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:42.777744 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb3d0e4-17d5-401e-b1aa-abe219260dbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=40636f0705234176be95cae0e97b604c, user_id=25016ba96e2144329ef5b62449974619, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:42.781485 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb3d0e4-17d5-401e-b1aa-abe219260dbc admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:42.797032 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1314/2649] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 15:17:42 2022] PUT /identity/v3/projects/40636f0705234176be95cae0e97b604c/users/25016ba96e2144329ef5b62449974619/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:17:42.808923 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7451bc-b418-4cae-b8e6-c35eeb2bfe97 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:42.809268 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7451bc-b418-4cae-b8e6-c35eeb2bfe97 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:42.810466 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7451bc-b418-4cae-b8e6-c35eeb2bfe97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:42.841561 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-1b7451bc-b418-4cae-b8e6-c35eeb2bfe97 None None] MFA Rules not processed for user `25016ba96e2144329ef5b62449974619`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:17:42.843058 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-1b7451bc-b418-4cae-b8e6-c35eeb2bfe97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:42.865806 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:42.919879 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1336/2650] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:17:42 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:17:42.924487 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb3d0e4-17d5-401e-b1aa-abe219260dbc admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:42.997586 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb3d0e4-17d5-401e-b1aa-abe219260dbc admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:43.002678 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe00f32-9ec7-437f-911c-ecb9513a6924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:43.003965 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe00f32-9ec7-437f-911c-ecb9513a6924 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:43.003965 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe00f32-9ec7-437f-911c-ecb9513a6924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:43.003965 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe00f32-9ec7-437f-911c-ecb9513a6924 admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:43.004682 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe00f32-9ec7-437f-911c-ecb9513a6924 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:43.006950 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe00f32-9ec7-437f-911c-ecb9513a6924 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:43.045604 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1315/2651] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 15:17:42 2022] POST /identity/v3/projects => generated 405 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:17:43.049957 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7451bc-b418-4cae-b8e6-c35eeb2bfe97 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:43.061598 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7451bc-b418-4cae-b8e6-c35eeb2bfe97 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:43.121604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4143c9d9-6d7e-4ee5-bbef-852bf703d994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:43.122900 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4143c9d9-6d7e-4ee5-bbef-852bf703d994 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:43.123223 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4143c9d9-6d7e-4ee5-bbef-852bf703d994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:43.123520 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4143c9d9-6d7e-4ee5-bbef-852bf703d994 admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:43.124186 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4143c9d9-6d7e-4ee5-bbef-852bf703d994 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:43.126138 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4143c9d9-6d7e-4ee5-bbef-852bf703d994 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:43.146106 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1337/2652] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 15:17:43 2022] POST /identity/v3/users => generated 309 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:17:43.150337 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe00f32-9ec7-437f-911c-ecb9513a6924 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:43.160894 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe00f32-9ec7-437f-911c-ecb9513a6924 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:43.165621 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea70055-bf32-4d11-9621-ecd12398d8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:43.166215 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea70055-bf32-4d11-9621-ecd12398d8e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:43.166495 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea70055-bf32-4d11-9621-ecd12398d8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:43.166769 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea70055-bf32-4d11-9621-ecd12398d8e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:43.167446 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea70055-bf32-4d11-9621-ecd12398d8e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:43.169607 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea70055-bf32-4d11-9621-ecd12398d8e7 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:43.181820 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1316/2653] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:17:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:17:43.186708 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4143c9d9-6d7e-4ee5-bbef-852bf703d994 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:43.197503 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4143c9d9-6d7e-4ee5-bbef-852bf703d994 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:43.203249 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68cb394-f56d-4be0-8e20-2b505e251564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:43.203909 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68cb394-f56d-4be0-8e20-2b505e251564 admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:43.204191 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68cb394-f56d-4be0-8e20-2b505e251564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:43.204461 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68cb394-f56d-4be0-8e20-2b505e251564 admin admin] PATH_INFO: `/v3/projects/573e379f6f5d4f6c8c26297703905e27/users/c6a95f5f89ca43b29b620e42c5d5d315/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:43.221212 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68cb394-f56d-4be0-8e20-2b505e251564 admin admin] RBAC: Authorizing `identity:create_grant(project_id=573e379f6f5d4f6c8c26297703905e27, user_id=c6a95f5f89ca43b29b620e42c5d5d315, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:43.224513 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68cb394-f56d-4be0-8e20-2b505e251564 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:43.238668 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1338/2654] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 15:17:43 2022] PUT /identity/v3/projects/573e379f6f5d4f6c8c26297703905e27/users/c6a95f5f89ca43b29b620e42c5d5d315/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:17:43.242817 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea70055-bf32-4d11-9621-ecd12398d8e7 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:43.253237 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea70055-bf32-4d11-9621-ecd12398d8e7 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:43.262186 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:43.315393 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb943357-c670-44c2-93ff-c77d2f0a94d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:43.316008 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb943357-c670-44c2-93ff-c77d2f0a94d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:43.316405 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb943357-c670-44c2-93ff-c77d2f0a94d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:43.316684 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb943357-c670-44c2-93ff-c77d2f0a94d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:43.317331 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb943357-c670-44c2-93ff-c77d2f0a94d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:43.318942 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb943357-c670-44c2-93ff-c77d2f0a94d4 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:43.330427 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1317/2655] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:17:43 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 15:17:43.334621 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68cb394-f56d-4be0-8e20-2b505e251564 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:43.345371 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68cb394-f56d-4be0-8e20-2b505e251564 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:43.350430 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81145f18-d9f0-4588-9b39-ce065e6ab0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:43.351076 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81145f18-d9f0-4588-9b39-ce065e6ab0b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:43.351393 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81145f18-d9f0-4588-9b39-ce065e6ab0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:43.351667 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81145f18-d9f0-4588-9b39-ce065e6ab0b2 admin admin] PATH_INFO: `/v3/projects/573e379f6f5d4f6c8c26297703905e27/users/c6a95f5f89ca43b29b620e42c5d5d315/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:43.353581 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81145f18-d9f0-4588-9b39-ce065e6ab0b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=573e379f6f5d4f6c8c26297703905e27, user_id=c6a95f5f89ca43b29b620e42c5d5d315, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:43.356226 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81145f18-d9f0-4588-9b39-ce065e6ab0b2 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:43.368333 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1339/2656] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 15:17:43 2022] PUT /identity/v3/projects/573e379f6f5d4f6c8c26297703905e27/users/c6a95f5f89ca43b29b620e42c5d5d315/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:17:43.379303 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978cac64-d797-4c36-ae25-b79e8a6d19dd None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:43.379763 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978cac64-d797-4c36-ae25-b79e8a6d19dd None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:43.379896 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978cac64-d797-4c36-ae25-b79e8a6d19dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:43.394578 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81145f18-d9f0-4588-9b39-ce065e6ab0b2 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:43.396912 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-81145f18-d9f0-4588-9b39-ce065e6ab0b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:43.416193 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-978cac64-d797-4c36-ae25-b79e8a6d19dd None None] MFA Rules not processed for user `c6a95f5f89ca43b29b620e42c5d5d315`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:17:43.417641 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-978cac64-d797-4c36-ae25-b79e8a6d19dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:43.502561 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81145f18-d9f0-4588-9b39-ce065e6ab0b2 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:43.506743 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1318/2657] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:17:43 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:17:43.509717 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb6c5a8-de63-4c72-9caf-1e2ef34b28b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:43.510655 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb6c5a8-de63-4c72-9caf-1e2ef34b28b0 service nova] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:43.510942 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb6c5a8-de63-4c72-9caf-1e2ef34b28b0 service nova] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:43.511228 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb6c5a8-de63-4c72-9caf-1e2ef34b28b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:43.513138 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-9eb6c5a8-de63-4c72-9caf-1e2ef34b28b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:43.518378 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978cac64-d797-4c36-ae25-b79e8a6d19dd None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:43.526840 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb6c5a8-de63-4c72-9caf-1e2ef34b28b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:43.529188 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb6c5a8-de63-4c72-9caf-1e2ef34b28b0 service nova] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:43.534350 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:43.541427 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1340/2658] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:17:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 15:17:43.579991 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978cac64-d797-4c36-ae25-b79e8a6d19dd None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:43.583866 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d2f813-2bc9-4c38-b97d-305a29a087c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:43.585067 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2f813-2bc9-4c38-b97d-305a29a087c4 service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:43.585416 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2f813-2bc9-4c38-b97d-305a29a087c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:43.586255 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2f813-2bc9-4c38-b97d-305a29a087c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:43.596694 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d2f813-2bc9-4c38-b97d-305a29a087c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:43.598134 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d2f813-2bc9-4c38-b97d-305a29a087c4 service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:43.608698 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1319/2659] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:17:43 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 15:17:43.764162 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb6c5a8-de63-4c72-9caf-1e2ef34b28b0 service nova] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:43.829103 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb6c5a8-de63-4c72-9caf-1e2ef34b28b0 service nova] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:43.833752 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec38cb-8412-46ce-8aec-5ac96ec3d65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:43.834472 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec38cb-8412-46ce-8aec-5ac96ec3d65e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:43.834745 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec38cb-8412-46ce-8aec-5ac96ec3d65e admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:43.835036 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec38cb-8412-46ce-8aec-5ac96ec3d65e admin admin] PATH_INFO: `/v3/users/25016ba96e2144329ef5b62449974619` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:43.836222 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ec38cb-8412-46ce-8aec-5ac96ec3d65e admin admin] RBAC: Authorizing `identity:delete_user(user_id=25016ba96e2144329ef5b62449974619)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:43.838311 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ec38cb-8412-46ce-8aec-5ac96ec3d65e admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:43.889304 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-63ec38cb-8412-46ce-8aec-5ac96ec3d65e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25016ba96e2144329ef5b62449974619'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:43.889974 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-63ec38cb-8412-46ce-8aec-5ac96ec3d65e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25016ba96e2144329ef5b62449974619'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:43.896781 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-63ec38cb-8412-46ce-8aec-5ac96ec3d65e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25016ba96e2144329ef5b62449974619'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:43.904809 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-63ec38cb-8412-46ce-8aec-5ac96ec3d65e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25016ba96e2144329ef5b62449974619'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:43.905476 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-63ec38cb-8412-46ce-8aec-5ac96ec3d65e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25016ba96e2144329ef5b62449974619'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:43.922934 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1341/2660] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 15:17:43 2022] DELETE /identity/v3/users/25016ba96e2144329ef5b62449974619 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:17:43.927661 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a732d0f-2e21-4dcd-8329-d128b8ccb7d7 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:43.928015 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a732d0f-2e21-4dcd-8329-d128b8ccb7d7 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:43.928362 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a732d0f-2e21-4dcd-8329-d128b8ccb7d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:43.950357 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-6a732d0f-2e21-4dcd-8329-d128b8ccb7d7 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:17:43.952326 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-6a732d0f-2e21-4dcd-8329-d128b8ccb7d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:43.989801 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:44.017370 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1320/2661] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:17:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:17:44.026837 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec38cb-8412-46ce-8aec-5ac96ec3d65e admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:44.028401 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-63ec38cb-8412-46ce-8aec-5ac96ec3d65e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:44.091112 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec38cb-8412-46ce-8aec-5ac96ec3d65e admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:44.094872 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb4e86d-228f-4735-b589-20c8727d882d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:44.095585 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4e86d-228f-4735-b589-20c8727d882d service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:44.095940 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4e86d-228f-4735-b589-20c8727d882d service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:44.096282 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4e86d-228f-4735-b589-20c8727d882d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:44.105865 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb4e86d-228f-4735-b589-20c8727d882d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:44.107631 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb4e86d-228f-4735-b589-20c8727d882d service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:44.117630 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1342/2662] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:17:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:17:44.411189 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a732d0f-2e21-4dcd-8329-d128b8ccb7d7 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:44.412993 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-6a732d0f-2e21-4dcd-8329-d128b8ccb7d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:44.428844 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a732d0f-2e21-4dcd-8329-d128b8ccb7d7 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:44.442485 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa193f9-9a35-48e1-9089-c9388fb92cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:44.443119 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa193f9-9a35-48e1-9089-c9388fb92cf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:44.443317 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa193f9-9a35-48e1-9089-c9388fb92cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:44.443658 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa193f9-9a35-48e1-9089-c9388fb92cf7 admin admin] PATH_INFO: `/v3/projects/40636f0705234176be95cae0e97b604c` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:44.444830 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa193f9-9a35-48e1-9089-c9388fb92cf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40636f0705234176be95cae0e97b604c)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:44.446717 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa193f9-9a35-48e1-9089-c9388fb92cf7 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:44.499672 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-cfa193f9-9a35-48e1-9089-c9388fb92cf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40636f0705234176be95cae0e97b604c', 'request_id': 'req-cfa193f9-9a35-48e1-9089-c9388fb92cf7'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:44.505121 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-cfa193f9-9a35-48e1-9089-c9388fb92cf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40636f0705234176be95cae0e97b604c', 'request_id': 'req-cfa193f9-9a35-48e1-9089-c9388fb92cf7'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:44.511354 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-cfa193f9-9a35-48e1-9089-c9388fb92cf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40636f0705234176be95cae0e97b604c', 'request_id': 'req-cfa193f9-9a35-48e1-9089-c9388fb92cf7'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:44.524162 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-cfa193f9-9a35-48e1-9089-c9388fb92cf7 admin admin] The token cache is being invalidate because project 40636f0705234176be95cae0e97b604c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:17:44.524331 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-cfa193f9-9a35-48e1-9089-c9388fb92cf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:44.525110 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-cfa193f9-9a35-48e1-9089-c9388fb92cf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:44.526864 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1321/2663] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 15:17:44 2022] DELETE /identity/v3/projects/40636f0705234176be95cae0e97b604c => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:17:44.531173 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb4e86d-228f-4735-b589-20c8727d882d service neutron] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:44.541264 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb4e86d-228f-4735-b589-20c8727d882d service neutron] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:44.558833 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:44.607896 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1443af67-b918-453b-8293-4824b0dbc61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:44.608522 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443af67-b918-453b-8293-4824b0dbc61d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:44.608796 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443af67-b918-453b-8293-4824b0dbc61d admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:44.609070 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443af67-b918-453b-8293-4824b0dbc61d admin admin] PATH_INFO: `/v3/users/c6a95f5f89ca43b29b620e42c5d5d315` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:44.610183 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1443af67-b918-453b-8293-4824b0dbc61d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6a95f5f89ca43b29b620e42c5d5d315)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:44.612185 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1443af67-b918-453b-8293-4824b0dbc61d admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:44.671343 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-1443af67-b918-453b-8293-4824b0dbc61d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6a95f5f89ca43b29b620e42c5d5d315'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:44.673715 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-1443af67-b918-453b-8293-4824b0dbc61d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6a95f5f89ca43b29b620e42c5d5d315'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:44.680103 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-1443af67-b918-453b-8293-4824b0dbc61d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6a95f5f89ca43b29b620e42c5d5d315'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:44.688186 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-1443af67-b918-453b-8293-4824b0dbc61d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6a95f5f89ca43b29b620e42c5d5d315'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:44.688793 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-1443af67-b918-453b-8293-4824b0dbc61d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6a95f5f89ca43b29b620e42c5d5d315'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:44.708843 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1343/2664] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 15:17:44 2022] DELETE /identity/v3/users/c6a95f5f89ca43b29b620e42c5d5d315 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:17:45.065553 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa193f9-9a35-48e1-9089-c9388fb92cf7 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:45.066981 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-cfa193f9-9a35-48e1-9089-c9388fb92cf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:45.088763 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa193f9-9a35-48e1-9089-c9388fb92cf7 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:45.156820 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e22de85-cd4b-4564-94be-e3487cff184f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:45.157453 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22de85-cd4b-4564-94be-e3487cff184f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:45.157730 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22de85-cd4b-4564-94be-e3487cff184f admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:45.157998 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22de85-cd4b-4564-94be-e3487cff184f admin admin] PATH_INFO: `/v3/projects/573e379f6f5d4f6c8c26297703905e27` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:45.159112 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e22de85-cd4b-4564-94be-e3487cff184f admin admin] RBAC: Authorizing `identity:delete_project(project_id=573e379f6f5d4f6c8c26297703905e27)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:45.161030 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e22de85-cd4b-4564-94be-e3487cff184f admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:45.209875 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-7e22de85-cd4b-4564-94be-e3487cff184f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '573e379f6f5d4f6c8c26297703905e27', 'request_id': 'req-7e22de85-cd4b-4564-94be-e3487cff184f'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:45.217935 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-7e22de85-cd4b-4564-94be-e3487cff184f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '573e379f6f5d4f6c8c26297703905e27', 'request_id': 'req-7e22de85-cd4b-4564-94be-e3487cff184f'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:45.225401 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-7e22de85-cd4b-4564-94be-e3487cff184f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '573e379f6f5d4f6c8c26297703905e27', 'request_id': 'req-7e22de85-cd4b-4564-94be-e3487cff184f'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:45.228220 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:45.241018 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-7e22de85-cd4b-4564-94be-e3487cff184f admin admin] The token cache is being invalidate because project 573e379f6f5d4f6c8c26297703905e27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:17:45.241305 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-7e22de85-cd4b-4564-94be-e3487cff184f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:45.241916 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-7e22de85-cd4b-4564-94be-e3487cff184f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:45.243453 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1322/2665] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 15:17:45 2022] DELETE /identity/v3/projects/573e379f6f5d4f6c8c26297703905e27 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:17:45.248786 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d5305-549d-47e6-ba4e-969579cbed28 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:45.249109 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d5305-549d-47e6-ba4e-969579cbed28 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:45.249381 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d5305-549d-47e6-ba4e-969579cbed28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:45.269472 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-d69d5305-549d-47e6-ba4e-969579cbed28 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:17:45.270671 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-d69d5305-549d-47e6-ba4e-969579cbed28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:45.329860 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1344/2666] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:17:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:17:45.337337 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede72df-c240-4de4-bbc0-469980d83c5e None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:45.337617 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede72df-c240-4de4-bbc0-469980d83c5e None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:45.337886 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede72df-c240-4de4-bbc0-469980d83c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:45.357749 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-cede72df-c240-4de4-bbc0-469980d83c5e None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:17:45.360231 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-cede72df-c240-4de4-bbc0-469980d83c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:45.369038 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1323/2667] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:17:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:17:45.373090 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69d5305-549d-47e6-ba4e-969579cbed28 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:45.391011 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69d5305-549d-47e6-ba4e-969579cbed28 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:45.395804 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcb7691-d9ce-467a-99ab-eb8e50e236e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:45.396414 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb7691-d9ce-467a-99ab-eb8e50e236e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:45.396698 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb7691-d9ce-467a-99ab-eb8e50e236e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:45.396972 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb7691-d9ce-467a-99ab-eb8e50e236e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:45.397626 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcb7691-d9ce-467a-99ab-eb8e50e236e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:17:45.397928 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcb7691-d9ce-467a-99ab-eb8e50e236e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:45.399564 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcb7691-d9ce-467a-99ab-eb8e50e236e0 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:45.413041 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1345/2668] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 15:17:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:17:45.417596 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cede72df-c240-4de4-bbc0-469980d83c5e None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:45.429040 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cede72df-c240-4de4-bbc0-469980d83c5e None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:45.434091 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63b40ac-f866-4932-a06b-ec8cfc6ad8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:45.435251 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b40ac-f866-4932-a06b-ec8cfc6ad8d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:45.435531 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b40ac-f866-4932-a06b-ec8cfc6ad8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:45.435796 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b40ac-f866-4932-a06b-ec8cfc6ad8d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:45.436423 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63b40ac-f866-4932-a06b-ec8cfc6ad8d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:45.438694 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63b40ac-f866-4932-a06b-ec8cfc6ad8d9 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:45.470974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1324/2669] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 15:17:45 2022] POST /identity/v3/projects => generated 369 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:17:45.475113 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcb7691-d9ce-467a-99ab-eb8e50e236e0 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:45.486506 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcb7691-d9ce-467a-99ab-eb8e50e236e0 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:45.510636 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:45.543339 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f956cfc5-5092-412f-9541-6b807e659b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:45.544027 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956cfc5-5092-412f-9541-6b807e659b2f admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:45.544796 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956cfc5-5092-412f-9541-6b807e659b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:45.545073 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956cfc5-5092-412f-9541-6b807e659b2f admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:45.545710 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f956cfc5-5092-412f-9541-6b807e659b2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:45.547651 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f956cfc5-5092-412f-9541-6b807e659b2f admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:45.570276 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1346/2670] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 15:17:45 2022] POST /identity/v3/users => generated 285 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:17:45.574641 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63b40ac-f866-4932-a06b-ec8cfc6ad8d9 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:45.585509 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63b40ac-f866-4932-a06b-ec8cfc6ad8d9 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:45.592716 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f7b024-ea41-415a-91c5-aa816ffe8c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:45.593391 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f7b024-ea41-415a-91c5-aa816ffe8c6b admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:45.593624 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f7b024-ea41-415a-91c5-aa816ffe8c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:45.593956 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f7b024-ea41-415a-91c5-aa816ffe8c6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:45.594647 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f7b024-ea41-415a-91c5-aa816ffe8c6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:45.596183 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f7b024-ea41-415a-91c5-aa816ffe8c6b admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:45.607435 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1325/2671] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:17:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:17:45.611406 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f956cfc5-5092-412f-9541-6b807e659b2f admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:45.621318 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f956cfc5-5092-412f-9541-6b807e659b2f admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:45.626035 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6048e7-6bf9-4a13-9893-c99d4fcc8ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:45.626621 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6048e7-6bf9-4a13-9893-c99d4fcc8ee4 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:45.626898 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6048e7-6bf9-4a13-9893-c99d4fcc8ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:45.627184 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6048e7-6bf9-4a13-9893-c99d4fcc8ee4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:45.627906 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6048e7-6bf9-4a13-9893-c99d4fcc8ee4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:45.629481 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6048e7-6bf9-4a13-9893-c99d4fcc8ee4 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:45.639898 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1347/2672] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:17:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:17:45.643911 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f7b024-ea41-415a-91c5-aa816ffe8c6b admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:45.654510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f7b024-ea41-415a-91c5-aa816ffe8c6b admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:45.659181 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544849b0-166a-4aa6-adb0-58936dae1068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:45.659921 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544849b0-166a-4aa6-adb0-58936dae1068 admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:45.660205 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544849b0-166a-4aa6-adb0-58936dae1068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:45.660484 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544849b0-166a-4aa6-adb0-58936dae1068 admin admin] PATH_INFO: `/v3/projects/13a2edd3147447ada62a51b74bd9dca4/users/da5be4a6c329419ea316904caad76be1/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:45.676738 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544849b0-166a-4aa6-adb0-58936dae1068 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13a2edd3147447ada62a51b74bd9dca4, user_id=da5be4a6c329419ea316904caad76be1, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:45.679567 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544849b0-166a-4aa6-adb0-58936dae1068 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:45.693759 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1326/2673] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 15:17:45 2022] PUT /identity/v3/projects/13a2edd3147447ada62a51b74bd9dca4/users/da5be4a6c329419ea316904caad76be1/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:17:45.703727 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a753089-1c0c-4cc8-80fa-bc989f011215 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:45.704009 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a753089-1c0c-4cc8-80fa-bc989f011215 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:45.704277 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a753089-1c0c-4cc8-80fa-bc989f011215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:45.733600 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-5a753089-1c0c-4cc8-80fa-bc989f011215 None None] MFA Rules not processed for user `da5be4a6c329419ea316904caad76be1`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:17:45.734793 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-5a753089-1c0c-4cc8-80fa-bc989f011215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:45.798672 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1348/2674] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:17:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:17:45.803054 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544849b0-166a-4aa6-adb0-58936dae1068 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:45.818431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a753089-1c0c-4cc8-80fa-bc989f011215 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:45.819931 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-5a753089-1c0c-4cc8-80fa-bc989f011215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:45.865735 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544849b0-166a-4aa6-adb0-58936dae1068 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:45.870371 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21901afa-f0f8-42f3-b6af-2744c9960105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:45.870966 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21901afa-f0f8-42f3-b6af-2744c9960105 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:45.871271 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21901afa-f0f8-42f3-b6af-2744c9960105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:45.871546 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21901afa-f0f8-42f3-b6af-2744c9960105 admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:45.872360 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21901afa-f0f8-42f3-b6af-2744c9960105 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:45.874266 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21901afa-f0f8-42f3-b6af-2744c9960105 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:45.885093 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a753089-1c0c-4cc8-80fa-bc989f011215 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:45.890302 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8e6d5f-d4d8-4092-b252-af92355e06c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:45.890953 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e6d5f-d4d8-4092-b252-af92355e06c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:45.891255 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e6d5f-d4d8-4092-b252-af92355e06c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:45.891527 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e6d5f-d4d8-4092-b252-af92355e06c9 admin admin] PATH_INFO: `/v3/users/0df8887828d541a2b5df73a28ee717a0` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:45.892723 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8e6d5f-d4d8-4092-b252-af92355e06c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0df8887828d541a2b5df73a28ee717a0)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:45.894682 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8e6d5f-d4d8-4092-b252-af92355e06c9 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:45.901475 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1327/2675] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 15:17:45 2022] POST /identity/v3/projects => generated 369 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:17:45.905985 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21901afa-f0f8-42f3-b6af-2744c9960105 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:45.935848 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:45.949181 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-db8e6d5f-d4d8-4092-b252-af92355e06c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0df8887828d541a2b5df73a28ee717a0'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:45.950671 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-db8e6d5f-d4d8-4092-b252-af92355e06c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0df8887828d541a2b5df73a28ee717a0'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:45.962471 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-db8e6d5f-d4d8-4092-b252-af92355e06c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0df8887828d541a2b5df73a28ee717a0'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:45.973240 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-db8e6d5f-d4d8-4092-b252-af92355e06c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0df8887828d541a2b5df73a28ee717a0'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:45.973969 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-db8e6d5f-d4d8-4092-b252-af92355e06c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0df8887828d541a2b5df73a28ee717a0'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:45.987038 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21901afa-f0f8-42f3-b6af-2744c9960105 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:45.992429 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e16259-eee0-453b-b012-69b8821ffcb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:45.993177 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e16259-eee0-453b-b012-69b8821ffcb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:45.993487 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e16259-eee0-453b-b012-69b8821ffcb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:45.993882 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e16259-eee0-453b-b012-69b8821ffcb8 admin admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:45.994599 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e16259-eee0-453b-b012-69b8821ffcb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:45.996816 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e16259-eee0-453b-b012-69b8821ffcb8 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:45.998198 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1349/2676] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 15:17:45 2022] DELETE /identity/v3/users/0df8887828d541a2b5df73a28ee717a0 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:17:46.002572 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e381a055-754e-42da-864a-ef3a4a5da4b9 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:46.003582 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e381a055-754e-42da-864a-ef3a4a5da4b9 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:46.003582 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e381a055-754e-42da-864a-ef3a4a5da4b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:46.020525 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1328/2677] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 15:17:45 2022] POST /identity/v3/users => generated 291 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:17:46.024696 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e16259-eee0-453b-b012-69b8821ffcb8 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:46.025550 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-e381a055-754e-42da-864a-ef3a4a5da4b9 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:17:46.026458 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-a3e16259-eee0-453b-b012-69b8821ffcb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:46.026755 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-e381a055-754e-42da-864a-ef3a4a5da4b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:46.039479 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:INSERT=7,keystone:DELETE=1,keystone:SELECT=92 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:46.046839 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e16259-eee0-453b-b012-69b8821ffcb8 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:46.112495 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1350/2678] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:17:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 15:17:46.120082 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e381a055-754e-42da-864a-ef3a4a5da4b9 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:46.122265 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-e381a055-754e-42da-864a-ef3a4a5da4b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:46.131163 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f20f5e6-8f30-4a74-9a2c-f3176f0ac4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:46.131960 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20f5e6-8f30-4a74-9a2c-f3176f0ac4a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:46.132343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20f5e6-8f30-4a74-9a2c-f3176f0ac4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:46.132714 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20f5e6-8f30-4a74-9a2c-f3176f0ac4a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:46.133460 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f20f5e6-8f30-4a74-9a2c-f3176f0ac4a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:46.135178 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f20f5e6-8f30-4a74-9a2c-f3176f0ac4a9 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:46.150602 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1329/2679] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 15:17:46.151718 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f20f5e6-8f30-4a74-9a2c-f3176f0ac4a9 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:46.163968 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f20f5e6-8f30-4a74-9a2c-f3176f0ac4a9 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:46.170964 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adf8846-a77b-4b45-8b20-0484f980c1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:46.171860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf8846-a77b-4b45-8b20-0484f980c1f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:46.172152 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf8846-a77b-4b45-8b20-0484f980c1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:46.172428 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf8846-a77b-4b45-8b20-0484f980c1f6 admin admin] PATH_INFO: `/v3/projects/289dc48d40f04ab795eb3b505d63b7f2/users/54521e13f7bb475d8f5fe5e199458633/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:46.198709 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adf8846-a77b-4b45-8b20-0484f980c1f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=289dc48d40f04ab795eb3b505d63b7f2, user_id=54521e13f7bb475d8f5fe5e199458633, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:46.202902 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adf8846-a77b-4b45-8b20-0484f980c1f6 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:46.220079 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e381a055-754e-42da-864a-ef3a4a5da4b9 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:46.229907 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1330/2680] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 15:17:46 2022] PUT /identity/v3/projects/289dc48d40f04ab795eb3b505d63b7f2/users/54521e13f7bb475d8f5fe5e199458633/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:17:46.240295 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adf8846-a77b-4b45-8b20-0484f980c1f6 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:46.283067 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74aee9f-da4e-4cd9-bfb5-bba807d7852f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:46.283873 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74aee9f-da4e-4cd9-bfb5-bba807d7852f service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:46.284154 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74aee9f-da4e-4cd9-bfb5-bba807d7852f service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:46.284428 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74aee9f-da4e-4cd9-bfb5-bba807d7852f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:46.312325 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:46.323562 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adf8846-a77b-4b45-8b20-0484f980c1f6 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:46.330860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e80d21-ef5d-4d4c-9acd-8eb3b4a5b2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:46.332050 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e80d21-ef5d-4d4c-9acd-8eb3b4a5b2eb admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:46.332482 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e80d21-ef5d-4d4c-9acd-8eb3b4a5b2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:46.332954 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e80d21-ef5d-4d4c-9acd-8eb3b4a5b2eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:46.333745 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e80d21-ef5d-4d4c-9acd-8eb3b4a5b2eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:46.335968 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e80d21-ef5d-4d4c-9acd-8eb3b4a5b2eb admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:46.356204 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1331/2681] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 15:17:46.358001 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e80d21-ef5d-4d4c-9acd-8eb3b4a5b2eb admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:46.370264 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74aee9f-da4e-4cd9-bfb5-bba807d7852f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:46.373550 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74aee9f-da4e-4cd9-bfb5-bba807d7852f service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:46.376415 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e80d21-ef5d-4d4c-9acd-8eb3b4a5b2eb admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:46.384145 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a34d194-ed0c-49b6-a691-1af07f3451df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:46.385056 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a34d194-ed0c-49b6-a691-1af07f3451df admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:46.385374 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a34d194-ed0c-49b6-a691-1af07f3451df admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:46.385685 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a34d194-ed0c-49b6-a691-1af07f3451df admin admin] PATH_INFO: `/v3/projects/289dc48d40f04ab795eb3b505d63b7f2/users/54521e13f7bb475d8f5fe5e199458633/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:46.388992 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a34d194-ed0c-49b6-a691-1af07f3451df admin admin] RBAC: Authorizing `identity:create_grant(project_id=289dc48d40f04ab795eb3b505d63b7f2, user_id=54521e13f7bb475d8f5fe5e199458633, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:46.391268 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1351/2682] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:17:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 15:17:46.392534 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a34d194-ed0c-49b6-a691-1af07f3451df admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:46.405046 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1332/2683] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 15:17:46 2022] PUT /identity/v3/projects/289dc48d40f04ab795eb3b505d63b7f2/users/54521e13f7bb475d8f5fe5e199458633/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:17:46.417287 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b6cc9-d6e8-41e2-99eb-6613b651a112 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:46.417624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b6cc9-d6e8-41e2-99eb-6613b651a112 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:46.418161 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b6cc9-d6e8-41e2-99eb-6613b651a112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:46.455849 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-238b6cc9-d6e8-41e2-99eb-6613b651a112 None None] MFA Rules not processed for user `54521e13f7bb475d8f5fe5e199458633`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:17:46.457630 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-238b6cc9-d6e8-41e2-99eb-6613b651a112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:46.535531 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1352/2684] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:17:46 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:17:46.548921 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a34d194-ed0c-49b6-a691-1af07f3451df admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:46.550694 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-8a34d194-ed0c-49b6-a691-1af07f3451df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:46.613857 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a34d194-ed0c-49b6-a691-1af07f3451df admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:46.617537 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c6f8c-10c9-4e1e-81a6-b65e3a6a9a66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:46.618146 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c6f8c-10c9-4e1e-81a6-b65e3a6a9a66 service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:46.618425 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c6f8c-10c9-4e1e-81a6-b65e3a6a9a66 service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:46.618698 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c6f8c-10c9-4e1e-81a6-b65e3a6a9a66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:46.629328 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24c6f8c-10c9-4e1e-81a6-b65e3a6a9a66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:46.630691 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24c6f8c-10c9-4e1e-81a6-b65e3a6a9a66 service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:46.636174 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:46.642165 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1333/2685] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:17:46 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:17:46.689994 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b6cc9-d6e8-41e2-99eb-6613b651a112 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:46.691606 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-238b6cc9-d6e8-41e2-99eb-6613b651a112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:46.704130 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b6cc9-d6e8-41e2-99eb-6613b651a112 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:46.707780 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea57720-4bbc-4619-a5b8-4510422a1a30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:46.708379 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea57720-4bbc-4619-a5b8-4510422a1a30 service cinder] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:46.708651 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea57720-4bbc-4619-a5b8-4510422a1a30 service cinder] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:46.708917 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea57720-4bbc-4619-a5b8-4510422a1a30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:46.711596 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-aea57720-4bbc-4619-a5b8-4510422a1a30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:46.779900 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea57720-4bbc-4619-a5b8-4510422a1a30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:46.781430 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea57720-4bbc-4619-a5b8-4510422a1a30 service cinder] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:46.790955 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1353/2686] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:17:46 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:17:46.900212 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c6f8c-10c9-4e1e-81a6-b65e3a6a9a66 service cinder] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:46.901789 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-d24c6f8c-10c9-4e1e-81a6-b65e3a6a9a66 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:46.993518 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c6f8c-10c9-4e1e-81a6-b65e3a6a9a66 service cinder] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:46.999087 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ce6d0-4a91-4ff5-b020-d93d8d0a291f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:46.999866 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ce6d0-4a91-4ff5-b020-d93d8d0a291f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:47.000284 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ce6d0-4a91-4ff5-b020-d93d8d0a291f admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:47.000729 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ce6d0-4a91-4ff5-b020-d93d8d0a291f admin admin] PATH_INFO: `/v3/projects/9260c0f22e10463294d486f5a442a3bc` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:47.002154 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ce6d0-4a91-4ff5-b020-d93d8d0a291f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9260c0f22e10463294d486f5a442a3bc)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:47.004282 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ce6d0-4a91-4ff5-b020-d93d8d0a291f admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:47.071749 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-c41ce6d0-4a91-4ff5-b020-d93d8d0a291f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9260c0f22e10463294d486f5a442a3bc', 'request_id': 'req-c41ce6d0-4a91-4ff5-b020-d93d8d0a291f'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:47.082284 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-c41ce6d0-4a91-4ff5-b020-d93d8d0a291f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9260c0f22e10463294d486f5a442a3bc', 'request_id': 'req-c41ce6d0-4a91-4ff5-b020-d93d8d0a291f'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:47.092561 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-c41ce6d0-4a91-4ff5-b020-d93d8d0a291f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9260c0f22e10463294d486f5a442a3bc', 'request_id': 'req-c41ce6d0-4a91-4ff5-b020-d93d8d0a291f'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:47.108841 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-c41ce6d0-4a91-4ff5-b020-d93d8d0a291f admin admin] The token cache is being invalidate because project 9260c0f22e10463294d486f5a442a3bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:17:47.109384 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-c41ce6d0-4a91-4ff5-b020-d93d8d0a291f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:47.110186 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-c41ce6d0-4a91-4ff5-b020-d93d8d0a291f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:47.115738 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1334/2687] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 15:17:46 2022] DELETE /identity/v3/projects/9260c0f22e10463294d486f5a442a3bc => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:17:47.118171 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ceeeca-5435-4aac-9140-2df974c87ab4 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:47.118809 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ceeeca-5435-4aac-9140-2df974c87ab4 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:47.119280 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ceeeca-5435-4aac-9140-2df974c87ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:47.145444 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-a8ceeeca-5435-4aac-9140-2df974c87ab4 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:17:47.146737 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-a8ceeeca-5435-4aac-9140-2df974c87ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:47.199734 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:47.225948 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1354/2688] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:17:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:17:47.235545 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198513b-774b-4c83-97d1-d8b0da57c52c None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:47.235905 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198513b-774b-4c83-97d1-d8b0da57c52c None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:47.236266 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198513b-774b-4c83-97d1-d8b0da57c52c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:47.256086 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-0198513b-774b-4c83-97d1-d8b0da57c52c None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:17:47.257434 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-0198513b-774b-4c83-97d1-d8b0da57c52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:47.267326 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1335/2689] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:17:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:17:47.271247 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ceeeca-5435-4aac-9140-2df974c87ab4 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:47.283021 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ceeeca-5435-4aac-9140-2df974c87ab4 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:47.291251 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d40d02-af4d-4c32-b407-a64a4bcf7b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:47.291951 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d40d02-af4d-4c32-b407-a64a4bcf7b95 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:47.292299 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d40d02-af4d-4c32-b407-a64a4bcf7b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:47.292639 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d40d02-af4d-4c32-b407-a64a4bcf7b95 admin admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:47.293402 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d40d02-af4d-4c32-b407-a64a4bcf7b95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:17:47.294752 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d40d02-af4d-4c32-b407-a64a4bcf7b95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:47.296528 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d40d02-af4d-4c32-b407-a64a4bcf7b95 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:47.311640 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1355/2690] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 15:17:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:17:47.316218 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0198513b-774b-4c83-97d1-d8b0da57c52c None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:47.327598 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0198513b-774b-4c83-97d1-d8b0da57c52c None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:47.332736 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8789ebe8-e39b-4ef3-b271-e5cc3c370964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:47.333433 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8789ebe8-e39b-4ef3-b271-e5cc3c370964 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:47.333704 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8789ebe8-e39b-4ef3-b271-e5cc3c370964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:47.333980 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8789ebe8-e39b-4ef3-b271-e5cc3c370964 admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:47.334635 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8789ebe8-e39b-4ef3-b271-e5cc3c370964 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:47.337097 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8789ebe8-e39b-4ef3-b271-e5cc3c370964 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:47.353462 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d40d02-af4d-4c32-b407-a64a4bcf7b95 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:47.356794 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-66d40d02-af4d-4c32-b407-a64a4bcf7b95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:47.369929 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1336/2691] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 15:17:47 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:17:47.374544 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8789ebe8-e39b-4ef3-b271-e5cc3c370964 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:47.424905 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d40d02-af4d-4c32-b407-a64a4bcf7b95 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:47.448381 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8789ebe8-e39b-4ef3-b271-e5cc3c370964 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:47.453731 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432c6ce7-795d-4176-8122-cf5b91f98d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:47.454465 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c6ce7-795d-4176-8122-cf5b91f98d37 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:47.454819 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c6ce7-795d-4176-8122-cf5b91f98d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:47.455163 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c6ce7-795d-4176-8122-cf5b91f98d37 admin admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:47.455910 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432c6ce7-795d-4176-8122-cf5b91f98d37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:47.457951 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432c6ce7-795d-4176-8122-cf5b91f98d37 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:47.468418 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:47.485380 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1337/2692] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 15:17:47 2022] POST /identity/v3/users => generated 287 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:17:47.489851 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432c6ce7-795d-4176-8122-cf5b91f98d37 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:47.491977 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6328ebaf-3ee7-4725-af2d-6912d3b13707 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:47.492852 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328ebaf-3ee7-4725-af2d-6912d3b13707 service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:47.494738 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328ebaf-3ee7-4725-af2d-6912d3b13707 service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:47.495394 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328ebaf-3ee7-4725-af2d-6912d3b13707 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:47.499759 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-6328ebaf-3ee7-4725-af2d-6912d3b13707 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:47.501848 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432c6ce7-795d-4176-8122-cf5b91f98d37 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:47.506833 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a46cb-0711-4d4e-9a0c-aa309dff1e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:47.507479 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a46cb-0711-4d4e-9a0c-aa309dff1e27 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:47.507748 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a46cb-0711-4d4e-9a0c-aa309dff1e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:47.508020 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a46cb-0711-4d4e-9a0c-aa309dff1e27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:47.508669 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4a46cb-0711-4d4e-9a0c-aa309dff1e27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:47.510280 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4a46cb-0711-4d4e-9a0c-aa309dff1e27 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:47.515891 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6328ebaf-3ee7-4725-af2d-6912d3b13707 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:47.518332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6328ebaf-3ee7-4725-af2d-6912d3b13707 service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:47.524032 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1338/2693] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:17:47.526185 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a46cb-0711-4d4e-9a0c-aa309dff1e27 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:47.533517 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1356/2694] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:17:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:17:47.539861 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a46cb-0711-4d4e-9a0c-aa309dff1e27 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:47.545540 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fda7063-9973-4ceb-89c2-f86b65c8c4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:47.546198 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda7063-9973-4ceb-89c2-f86b65c8c4bf admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:47.546480 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda7063-9973-4ceb-89c2-f86b65c8c4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:47.546752 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda7063-9973-4ceb-89c2-f86b65c8c4bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:47.547437 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fda7063-9973-4ceb-89c2-f86b65c8c4bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:47.549065 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fda7063-9973-4ceb-89c2-f86b65c8c4bf admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:47.564119 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1339/2695] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:17:47.565422 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6328ebaf-3ee7-4725-af2d-6912d3b13707 service neutron] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:47.576099 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6328ebaf-3ee7-4725-af2d-6912d3b13707 service neutron] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:47.582552 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19222187-407e-4623-be17-68ab5531c557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:47.583328 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19222187-407e-4623-be17-68ab5531c557 admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:47.583658 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19222187-407e-4623-be17-68ab5531c557 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:47.583996 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19222187-407e-4623-be17-68ab5531c557 admin admin] PATH_INFO: `/v3/projects/7034e8a9461846a7b08fe7328ecf7a47/users/b2905d19c05b4f53ba70e2bb3d165bc1/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:47.603108 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19222187-407e-4623-be17-68ab5531c557 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7034e8a9461846a7b08fe7328ecf7a47, user_id=b2905d19c05b4f53ba70e2bb3d165bc1, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:47.608156 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19222187-407e-4623-be17-68ab5531c557 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:47.641653 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1357/2696] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 15:17:47 2022] PUT /identity/v3/projects/7034e8a9461846a7b08fe7328ecf7a47/users/b2905d19c05b4f53ba70e2bb3d165bc1/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:17:47.641955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c536e68-a997-4f07-9db6-6e5702e5c682 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:47.642245 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c536e68-a997-4f07-9db6-6e5702e5c682 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:47.642543 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c536e68-a997-4f07-9db6-6e5702e5c682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:47.664331 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-5c536e68-a997-4f07-9db6-6e5702e5c682 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:17:47.665667 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-5c536e68-a997-4f07-9db6-6e5702e5c682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:47.739954 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1340/2697] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:17:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:17:47.756565 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19222187-407e-4623-be17-68ab5531c557 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:47.848250 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19222187-407e-4623-be17-68ab5531c557 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:47.852342 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2297353f-5ab7-4d13-946a-66724d5d7fda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:47.852946 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2297353f-5ab7-4d13-946a-66724d5d7fda service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:47.853226 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2297353f-5ab7-4d13-946a-66724d5d7fda service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:47.853491 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2297353f-5ab7-4d13-946a-66724d5d7fda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:47.866422 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2297353f-5ab7-4d13-946a-66724d5d7fda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:47.868496 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2297353f-5ab7-4d13-946a-66724d5d7fda service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:47.879073 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1358/2698] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:17:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:17:48.334203 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c536e68-a997-4f07-9db6-6e5702e5c682 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:48.338597 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-5c536e68-a997-4f07-9db6-6e5702e5c682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:48.362246 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c536e68-a997-4f07-9db6-6e5702e5c682 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:48.371272 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c93f1-de8b-4e17-810b-05025db58887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:48.372053 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c93f1-de8b-4e17-810b-05025db58887 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:48.372333 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c93f1-de8b-4e17-810b-05025db58887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:48.374718 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c93f1-de8b-4e17-810b-05025db58887 admin admin] PATH_INFO: `/v3/users/6fac2f6b6c124ddf99a888aa0c30f83b` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:48.379776 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1c93f1-de8b-4e17-810b-05025db58887 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fac2f6b6c124ddf99a888aa0c30f83b)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:48.381859 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1c93f1-de8b-4e17-810b-05025db58887 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:48.472164 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-fd1c93f1-de8b-4e17-810b-05025db58887 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fac2f6b6c124ddf99a888aa0c30f83b'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:48.473866 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-fd1c93f1-de8b-4e17-810b-05025db58887 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fac2f6b6c124ddf99a888aa0c30f83b'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:48.495817 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-fd1c93f1-de8b-4e17-810b-05025db58887 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fac2f6b6c124ddf99a888aa0c30f83b'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:48.508850 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-fd1c93f1-de8b-4e17-810b-05025db58887 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fac2f6b6c124ddf99a888aa0c30f83b'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:48.520261 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-fd1c93f1-de8b-4e17-810b-05025db58887 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fac2f6b6c124ddf99a888aa0c30f83b'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:48.527056 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1341/2699] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 15:17:48 2022] DELETE /identity/v3/users/6fac2f6b6c124ddf99a888aa0c30f83b => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:17:48.795723 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2297353f-5ab7-4d13-946a-66724d5d7fda service neutron] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:48.809997 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2297353f-5ab7-4d13-946a-66724d5d7fda service neutron] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:48.839371 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:48.907059 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd152165-f8cc-4087-b46d-e4ebc8916626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:48.907761 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd152165-f8cc-4087-b46d-e4ebc8916626 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:48.908062 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd152165-f8cc-4087-b46d-e4ebc8916626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:48.908339 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd152165-f8cc-4087-b46d-e4ebc8916626 admin admin] PATH_INFO: `/v3/projects/1bfe11f6c0da4508b94a01afb80c8483` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:48.909490 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd152165-f8cc-4087-b46d-e4ebc8916626 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bfe11f6c0da4508b94a01afb80c8483)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:48.911433 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd152165-f8cc-4087-b46d-e4ebc8916626 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:48.964240 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-bd152165-f8cc-4087-b46d-e4ebc8916626 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bfe11f6c0da4508b94a01afb80c8483', 'request_id': 'req-bd152165-f8cc-4087-b46d-e4ebc8916626'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:48.972457 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-bd152165-f8cc-4087-b46d-e4ebc8916626 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bfe11f6c0da4508b94a01afb80c8483', 'request_id': 'req-bd152165-f8cc-4087-b46d-e4ebc8916626'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:48.986516 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-bd152165-f8cc-4087-b46d-e4ebc8916626 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bfe11f6c0da4508b94a01afb80c8483', 'request_id': 'req-bd152165-f8cc-4087-b46d-e4ebc8916626'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:49.005967 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-bd152165-f8cc-4087-b46d-e4ebc8916626 admin admin] The token cache is being invalidate because project 1bfe11f6c0da4508b94a01afb80c8483 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:17:49.006203 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-bd152165-f8cc-4087-b46d-e4ebc8916626 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:49.006825 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-bd152165-f8cc-4087-b46d-e4ebc8916626 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:49.023239 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1359/2700] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 15:17:48 2022] DELETE /identity/v3/projects/1bfe11f6c0da4508b94a01afb80c8483 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:17:50.469479 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c93f1-de8b-4e17-810b-05025db58887 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:50.471113 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-fd1c93f1-de8b-4e17-810b-05025db58887 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:50.485287 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c93f1-de8b-4e17-810b-05025db58887 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:50.549831 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:50.581708 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc6c199-6973-4ce1-9406-0cf9a08d7252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:50.582367 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6c199-6973-4ce1-9406-0cf9a08d7252 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:50.582981 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6c199-6973-4ce1-9406-0cf9a08d7252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:50.583439 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6c199-6973-4ce1-9406-0cf9a08d7252 admin admin] PATH_INFO: `/v3/users/da5be4a6c329419ea316904caad76be1` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:50.585250 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc6c199-6973-4ce1-9406-0cf9a08d7252 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da5be4a6c329419ea316904caad76be1)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:50.587567 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc6c199-6973-4ce1-9406-0cf9a08d7252 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:50.644433 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-6fc6c199-6973-4ce1-9406-0cf9a08d7252 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da5be4a6c329419ea316904caad76be1'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:50.645230 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-6fc6c199-6973-4ce1-9406-0cf9a08d7252 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da5be4a6c329419ea316904caad76be1'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:50.661786 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-6fc6c199-6973-4ce1-9406-0cf9a08d7252 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da5be4a6c329419ea316904caad76be1'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:50.674735 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-6fc6c199-6973-4ce1-9406-0cf9a08d7252 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da5be4a6c329419ea316904caad76be1'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:50.688052 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-6fc6c199-6973-4ce1-9406-0cf9a08d7252 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da5be4a6c329419ea316904caad76be1'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:50.697494 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1342/2701] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 15:17:50 2022] DELETE /identity/v3/users/da5be4a6c329419ea316904caad76be1 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:17:50.702281 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8e3176-2363-4d39-96ce-b3e225c8c203 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:50.702782 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8e3176-2363-4d39-96ce-b3e225c8c203 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:50.702930 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8e3176-2363-4d39-96ce-b3e225c8c203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:50.724167 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-5f8e3176-2363-4d39-96ce-b3e225c8c203 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:17:50.725952 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-5f8e3176-2363-4d39-96ce-b3e225c8c203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:50.798114 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1360/2702] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:17:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:17:50.811052 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc6c199-6973-4ce1-9406-0cf9a08d7252 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:50.812714 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-6fc6c199-6973-4ce1-9406-0cf9a08d7252 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:50.891718 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc6c199-6973-4ce1-9406-0cf9a08d7252 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:50.895633 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b46715d-8489-4845-b7b3-47b5fd610834 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:50.896254 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b46715d-8489-4845-b7b3-47b5fd610834 service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:50.896531 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b46715d-8489-4845-b7b3-47b5fd610834 service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:50.896799 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b46715d-8489-4845-b7b3-47b5fd610834 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:50.906924 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b46715d-8489-4845-b7b3-47b5fd610834 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:50.908403 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b46715d-8489-4845-b7b3-47b5fd610834 service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:50.919761 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1343/2703] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:17:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:17:51.322596 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8e3176-2363-4d39-96ce-b3e225c8c203 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:51.324143 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-5f8e3176-2363-4d39-96ce-b3e225c8c203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:51.346593 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8e3176-2363-4d39-96ce-b3e225c8c203 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:51.352445 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa80beb8-21c6-46e7-99a7-174a31f116fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:51.352949 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa80beb8-21c6-46e7-99a7-174a31f116fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:51.353227 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa80beb8-21c6-46e7-99a7-174a31f116fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:51.353495 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa80beb8-21c6-46e7-99a7-174a31f116fa admin admin] PATH_INFO: `/v3/projects/13a2edd3147447ada62a51b74bd9dca4` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:51.354619 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa80beb8-21c6-46e7-99a7-174a31f116fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=13a2edd3147447ada62a51b74bd9dca4)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:51.357115 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa80beb8-21c6-46e7-99a7-174a31f116fa admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:51.404459 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:51.417135 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-fa80beb8-21c6-46e7-99a7-174a31f116fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13a2edd3147447ada62a51b74bd9dca4', 'request_id': 'req-fa80beb8-21c6-46e7-99a7-174a31f116fa'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:51.424715 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-fa80beb8-21c6-46e7-99a7-174a31f116fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13a2edd3147447ada62a51b74bd9dca4', 'request_id': 'req-fa80beb8-21c6-46e7-99a7-174a31f116fa'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:51.432039 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-fa80beb8-21c6-46e7-99a7-174a31f116fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13a2edd3147447ada62a51b74bd9dca4', 'request_id': 'req-fa80beb8-21c6-46e7-99a7-174a31f116fa'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:51.450694 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-fa80beb8-21c6-46e7-99a7-174a31f116fa admin admin] The token cache is being invalidate because project 13a2edd3147447ada62a51b74bd9dca4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:17:51.451045 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-fa80beb8-21c6-46e7-99a7-174a31f116fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:51.451933 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-fa80beb8-21c6-46e7-99a7-174a31f116fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:51.455254 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1361/2704] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 15:17:51 2022] DELETE /identity/v3/projects/13a2edd3147447ada62a51b74bd9dca4 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:17:51.458092 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b46715d-8489-4845-b7b3-47b5fd610834 service neutron] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:51.470104 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b46715d-8489-4845-b7b3-47b5fd610834 service neutron] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:51.538481 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee25155c-e766-4f4b-8e80-0f7be35f7284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:51.539444 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25155c-e766-4f4b-8e80-0f7be35f7284 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:51.539888 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25155c-e766-4f4b-8e80-0f7be35f7284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:51.540203 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25155c-e766-4f4b-8e80-0f7be35f7284 admin admin] PATH_INFO: `/v3/users/54521e13f7bb475d8f5fe5e199458633` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:51.541523 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee25155c-e766-4f4b-8e80-0f7be35f7284 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54521e13f7bb475d8f5fe5e199458633)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:51.543844 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee25155c-e766-4f4b-8e80-0f7be35f7284 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:51.549993 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:51.606099 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-ee25155c-e766-4f4b-8e80-0f7be35f7284 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54521e13f7bb475d8f5fe5e199458633'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:51.606804 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-ee25155c-e766-4f4b-8e80-0f7be35f7284 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54521e13f7bb475d8f5fe5e199458633'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:51.622338 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-ee25155c-e766-4f4b-8e80-0f7be35f7284 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54521e13f7bb475d8f5fe5e199458633'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:51.631901 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-ee25155c-e766-4f4b-8e80-0f7be35f7284 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54521e13f7bb475d8f5fe5e199458633'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:51.641725 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-ee25155c-e766-4f4b-8e80-0f7be35f7284 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54521e13f7bb475d8f5fe5e199458633'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:51.653007 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1344/2705] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 15:17:51 2022] DELETE /identity/v3/users/54521e13f7bb475d8f5fe5e199458633 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:17:52.136598 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa80beb8-21c6-46e7-99a7-174a31f116fa admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:52.140404 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-fa80beb8-21c6-46e7-99a7-174a31f116fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:52.170055 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa80beb8-21c6-46e7-99a7-174a31f116fa admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:52.279859 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db43ec9-4e90-4817-b089-de5373227ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:52.280671 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db43ec9-4e90-4817-b089-de5373227ee7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:52.280944 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db43ec9-4e90-4817-b089-de5373227ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:52.281612 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db43ec9-4e90-4817-b089-de5373227ee7 admin admin] PATH_INFO: `/v3/projects/289dc48d40f04ab795eb3b505d63b7f2` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:52.282547 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db43ec9-4e90-4817-b089-de5373227ee7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=289dc48d40f04ab795eb3b505d63b7f2)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:52.284747 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db43ec9-4e90-4817-b089-de5373227ee7 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:52.342323 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-7db43ec9-4e90-4817-b089-de5373227ee7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '289dc48d40f04ab795eb3b505d63b7f2', 'request_id': 'req-7db43ec9-4e90-4817-b089-de5373227ee7'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:52.352457 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-7db43ec9-4e90-4817-b089-de5373227ee7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '289dc48d40f04ab795eb3b505d63b7f2', 'request_id': 'req-7db43ec9-4e90-4817-b089-de5373227ee7'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:52.360812 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-7db43ec9-4e90-4817-b089-de5373227ee7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '289dc48d40f04ab795eb3b505d63b7f2', 'request_id': 'req-7db43ec9-4e90-4817-b089-de5373227ee7'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:52.377323 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-7db43ec9-4e90-4817-b089-de5373227ee7 admin admin] The token cache is being invalidate because project 289dc48d40f04ab795eb3b505d63b7f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:17:52.377541 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-7db43ec9-4e90-4817-b089-de5373227ee7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:52.378751 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-7db43ec9-4e90-4817-b089-de5373227ee7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:52.383608 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1362/2706] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 15:17:52 2022] DELETE /identity/v3/projects/289dc48d40f04ab795eb3b505d63b7f2 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:17:52.386086 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e0632-d0ee-46ea-b5f5-7602adef9d57 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:52.386299 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e0632-d0ee-46ea-b5f5-7602adef9d57 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:52.387893 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e0632-d0ee-46ea-b5f5-7602adef9d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:52.419607 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-767e0632-d0ee-46ea-b5f5-7602adef9d57 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:17:52.420976 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-767e0632-d0ee-46ea-b5f5-7602adef9d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:52.499943 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1345/2707] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:17:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:17:52.507495 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076b202-b8d1-42ff-a5e7-72366d0cda9e None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:52.508028 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076b202-b8d1-42ff-a5e7-72366d0cda9e None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:52.508317 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076b202-b8d1-42ff-a5e7-72366d0cda9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:52.537801 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-1076b202-b8d1-42ff-a5e7-72366d0cda9e None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:17:52.538074 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-1076b202-b8d1-42ff-a5e7-72366d0cda9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:52.551633 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1363/2708] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:17:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:17:52.555780 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e0632-d0ee-46ea-b5f5-7602adef9d57 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:52.569860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e0632-d0ee-46ea-b5f5-7602adef9d57 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:52.580613 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b24998-8371-4419-b600-ebbd89042c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:52.581331 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b24998-8371-4419-b600-ebbd89042c7b admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:52.581959 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b24998-8371-4419-b600-ebbd89042c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:52.582306 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b24998-8371-4419-b600-ebbd89042c7b admin admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:52.583051 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b24998-8371-4419-b600-ebbd89042c7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:17:52.583455 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b24998-8371-4419-b600-ebbd89042c7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:52.586816 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b24998-8371-4419-b600-ebbd89042c7b admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:52.609230 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1346/2709] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 15:17:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:17:52.613787 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076b202-b8d1-42ff-a5e7-72366d0cda9e None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:52.624789 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076b202-b8d1-42ff-a5e7-72366d0cda9e None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:52.625183 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae348b95-bc96-45c0-b539-522528bd323c None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:52.625470 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae348b95-bc96-45c0-b539-522528bd323c None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:52.625744 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae348b95-bc96-45c0-b539-522528bd323c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:52.629862 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e59b402-40eb-4241-b4fb-a36a6e7c83bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:52.630410 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e59b402-40eb-4241-b4fb-a36a6e7c83bf admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:52.630687 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e59b402-40eb-4241-b4fb-a36a6e7c83bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:52.630957 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e59b402-40eb-4241-b4fb-a36a6e7c83bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:52.631622 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e59b402-40eb-4241-b4fb-a36a6e7c83bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:52.633542 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e59b402-40eb-4241-b4fb-a36a6e7c83bf admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:52.660827 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-ae348b95-bc96-45c0-b539-522528bd323c None None] MFA Rules not processed for user `b2905d19c05b4f53ba70e2bb3d165bc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:17:52.662029 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-ae348b95-bc96-45c0-b539-522528bd323c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:52.664969 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1364/2710] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 15:17:52 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:17:52.669077 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e59b402-40eb-4241-b4fb-a36a6e7c83bf admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:52.703343 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:52.735641 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:52.741227 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1347/2711] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:17:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 15:17:52.751416 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae348b95-bc96-45c0-b539-522528bd323c None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:52.754068 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e59b402-40eb-4241-b4fb-a36a6e7c83bf admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:52.766588 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-ae348b95-bc96-45c0-b539-522528bd323c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:52.771188 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783ff71-349f-4e1b-a43e-8e5a44413eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:52.772285 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783ff71-349f-4e1b-a43e-8e5a44413eae admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:52.772830 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783ff71-349f-4e1b-a43e-8e5a44413eae admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:52.773104 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783ff71-349f-4e1b-a43e-8e5a44413eae admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:52.773742 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0783ff71-349f-4e1b-a43e-8e5a44413eae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:52.775686 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0783ff71-349f-4e1b-a43e-8e5a44413eae admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:52.786241 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae348b95-bc96-45c0-b539-522528bd323c None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:52.793053 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39e4533-8558-4747-972a-e8650883c0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:52.794738 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e4533-8558-4747-972a-e8650883c0a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:52.795162 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e4533-8558-4747-972a-e8650883c0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:52.795584 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e4533-8558-4747-972a-e8650883c0a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:52.796748 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39e4533-8558-4747-972a-e8650883c0a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:52.799107 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39e4533-8558-4747-972a-e8650883c0a9 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:52.808972 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1365/2712] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 15:17:52 2022] POST /identity/v3/users => generated 292 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:17:52.812940 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783ff71-349f-4e1b-a43e-8e5a44413eae admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:52.829686 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783ff71-349f-4e1b-a43e-8e5a44413eae admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:52.837194 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1348/2713] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 15:17:52 2022] POST /identity/v3/projects => generated 369 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:17:52.839380 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c406d03-8f7d-4d1f-ad27-bd49671cf886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:52.840234 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c406d03-8f7d-4d1f-ad27-bd49671cf886 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:52.841154 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c406d03-8f7d-4d1f-ad27-bd49671cf886 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:52.841536 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c406d03-8f7d-4d1f-ad27-bd49671cf886 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:52.842041 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39e4533-8558-4747-972a-e8650883c0a9 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:52.842525 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c406d03-8f7d-4d1f-ad27-bd49671cf886 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:52.844367 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c406d03-8f7d-4d1f-ad27-bd49671cf886 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:52.855929 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1366/2714] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:17:52 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 15:17:52.859927 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c406d03-8f7d-4d1f-ad27-bd49671cf886 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:52.874762 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c406d03-8f7d-4d1f-ad27-bd49671cf886 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:52.923384 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39e4533-8558-4747-972a-e8650883c0a9 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:52.930605 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a311bc4a-11e0-4fc7-93fc-955c0fac697e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:52.931751 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311bc4a-11e0-4fc7-93fc-955c0fac697e admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:52.932105 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311bc4a-11e0-4fc7-93fc-955c0fac697e admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:52.932476 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311bc4a-11e0-4fc7-93fc-955c0fac697e admin admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:52.933214 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a311bc4a-11e0-4fc7-93fc-955c0fac697e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:52.935860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a311bc4a-11e0-4fc7-93fc-955c0fac697e admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:52.948872 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f2a355-6fcf-4b47-8213-b83d26f0b691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:52.950319 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f2a355-6fcf-4b47-8213-b83d26f0b691 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:52.950319 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f2a355-6fcf-4b47-8213-b83d26f0b691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:52.950629 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f2a355-6fcf-4b47-8213-b83d26f0b691 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:52.951571 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f2a355-6fcf-4b47-8213-b83d26f0b691 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:52.953101 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f2a355-6fcf-4b47-8213-b83d26f0b691 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:52.962311 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1349/2715] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 15:17:52 2022] POST /identity/v3/users => generated 291 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:17:52.964169 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1367/2716] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:17:52 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 15:17:52.967725 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a311bc4a-11e0-4fc7-93fc-955c0fac697e admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:52.968755 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f2a355-6fcf-4b47-8213-b83d26f0b691 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:52.979479 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a311bc4a-11e0-4fc7-93fc-955c0fac697e admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:52.980172 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f2a355-6fcf-4b47-8213-b83d26f0b691 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:52.985928 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b34e20-3123-482e-bb4b-d04013bd7a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:52.986541 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac05a19-4fd6-4f06-8d2e-1b83dc14c0b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:52.986698 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b34e20-3123-482e-bb4b-d04013bd7a01 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:52.986845 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b34e20-3123-482e-bb4b-d04013bd7a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:52.987157 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b34e20-3123-482e-bb4b-d04013bd7a01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:52.987621 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac05a19-4fd6-4f06-8d2e-1b83dc14c0b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:52.987864 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b34e20-3123-482e-bb4b-d04013bd7a01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:52.987991 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac05a19-4fd6-4f06-8d2e-1b83dc14c0b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:52.988165 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac05a19-4fd6-4f06-8d2e-1b83dc14c0b8 admin admin] PATH_INFO: `/v3/projects/ad481ec6814944d798441af409f5fed7/users/113348f1776e45fe8ab331aea5ab010c/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:52.989424 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b34e20-3123-482e-bb4b-d04013bd7a01 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:53.000712 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1350/2717] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:17:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:17:53.006003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b34e20-3123-482e-bb4b-d04013bd7a01 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:53.007723 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac05a19-4fd6-4f06-8d2e-1b83dc14c0b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad481ec6814944d798441af409f5fed7, user_id=113348f1776e45fe8ab331aea5ab010c, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:53.011090 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac05a19-4fd6-4f06-8d2e-1b83dc14c0b8 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:53.016679 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b34e20-3123-482e-bb4b-d04013bd7a01 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:53.023877 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ba48be-d630-4a04-bf3b-31da20085de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:53.024831 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba48be-d630-4a04-bf3b-31da20085de8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:53.025255 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba48be-d630-4a04-bf3b-31da20085de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:53.025656 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba48be-d630-4a04-bf3b-31da20085de8 admin admin] PATH_INFO: `/v3/projects/be98d82b0c6b40439bee9af8733cc980/users/61fc09826a874f7f9a9d25238e5dceb8/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:53.030463 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1368/2718] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 15:17:52 2022] PUT /identity/v3/projects/ad481ec6814944d798441af409f5fed7/users/113348f1776e45fe8ab331aea5ab010c/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:17:53.041206 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc795a54-f79f-4c44-bad1-adba1eb94345 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:53.041488 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc795a54-f79f-4c44-bad1-adba1eb94345 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:53.041757 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc795a54-f79f-4c44-bad1-adba1eb94345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:53.046082 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ba48be-d630-4a04-bf3b-31da20085de8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be98d82b0c6b40439bee9af8733cc980, user_id=61fc09826a874f7f9a9d25238e5dceb8, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:53.049202 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ba48be-d630-4a04-bf3b-31da20085de8 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:53.065025 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1351/2719] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 15:17:53 2022] PUT /identity/v3/projects/be98d82b0c6b40439bee9af8733cc980/users/61fc09826a874f7f9a9d25238e5dceb8/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:17:53.069020 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ba48be-d630-4a04-bf3b-31da20085de8 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:53.073861 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-fc795a54-f79f-4c44-bad1-adba1eb94345 None None] MFA Rules not processed for user `113348f1776e45fe8ab331aea5ab010c`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:17:53.075480 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-fc795a54-f79f-4c44-bad1-adba1eb94345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:53.139750 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ba48be-d630-4a04-bf3b-31da20085de8 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:53.144628 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1993302f-fd96-490d-8136-0873784ca821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:53.145241 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1993302f-fd96-490d-8136-0873784ca821 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:53.145517 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1993302f-fd96-490d-8136-0873784ca821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:53.145785 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1993302f-fd96-490d-8136-0873784ca821 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:53.146624 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1993302f-fd96-490d-8136-0873784ca821 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:53.148633 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1993302f-fd96-490d-8136-0873784ca821 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:53.148795 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1369/2720] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:17:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:17:53.154254 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc795a54-f79f-4c44-bad1-adba1eb94345 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:53.161267 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:53.165263 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1352/2721] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 15:17:53.165754 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc795a54-f79f-4c44-bad1-adba1eb94345 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:53.170567 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1993302f-fd96-490d-8136-0873784ca821 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:53.172116 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66758aa-f4ab-4fb3-b6c3-e2514f3fb9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:53.172848 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66758aa-f4ab-4fb3-b6c3-e2514f3fb9ef admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:53.173135 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66758aa-f4ab-4fb3-b6c3-e2514f3fb9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:53.173410 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66758aa-f4ab-4fb3-b6c3-e2514f3fb9ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:53.174047 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66758aa-f4ab-4fb3-b6c3-e2514f3fb9ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:53.175983 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66758aa-f4ab-4fb3-b6c3-e2514f3fb9ef admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:53.181140 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1993302f-fd96-490d-8136-0873784ca821 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:53.186293 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9597f244-fe80-431f-a4a6-c5811c2ca0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:53.186997 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9597f244-fe80-431f-a4a6-c5811c2ca0fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:53.187426 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9597f244-fe80-431f-a4a6-c5811c2ca0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:53.187704 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9597f244-fe80-431f-a4a6-c5811c2ca0fe admin admin] PATH_INFO: `/v3/projects/be98d82b0c6b40439bee9af8733cc980/users/61fc09826a874f7f9a9d25238e5dceb8/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:53.189679 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9597f244-fe80-431f-a4a6-c5811c2ca0fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=be98d82b0c6b40439bee9af8733cc980, user_id=61fc09826a874f7f9a9d25238e5dceb8, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:53.193062 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9597f244-fe80-431f-a4a6-c5811c2ca0fe admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:53.211319 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1353/2722] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 15:17:53 2022] PUT /identity/v3/projects/be98d82b0c6b40439bee9af8733cc980/users/61fc09826a874f7f9a9d25238e5dceb8/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:17:53.220833 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1370/2723] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 15:17:53 2022] POST /identity/v3/projects => generated 381 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:17:53.222210 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9597f244-fe80-431f-a4a6-c5811c2ca0fe admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:53.252306 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:53.328155 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9597f244-fe80-431f-a4a6-c5811c2ca0fe admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:53.404240 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89d770-ae2d-4d1f-a2e4-7586eaeb9732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:53.404990 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89d770-ae2d-4d1f-a2e4-7586eaeb9732 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:53.405538 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89d770-ae2d-4d1f-a2e4-7586eaeb9732 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:53.405683 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89d770-ae2d-4d1f-a2e4-7586eaeb9732 admin admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:53.406368 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af89d770-ae2d-4d1f-a2e4-7586eaeb9732 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:53.408457 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af89d770-ae2d-4d1f-a2e4-7586eaeb9732 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:53.436682 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1354/2724] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 15:17:53 2022] POST /identity/v3/users => generated 297 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:17:53.441010 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66758aa-f4ab-4fb3-b6c3-e2514f3fb9ef admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:53.455068 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66758aa-f4ab-4fb3-b6c3-e2514f3fb9ef admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:53.461812 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69cf709-ed69-48fe-9b8e-365e69141104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:53.462495 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69cf709-ed69-48fe-9b8e-365e69141104 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:53.462831 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69cf709-ed69-48fe-9b8e-365e69141104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:53.463417 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69cf709-ed69-48fe-9b8e-365e69141104 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:53.464131 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69cf709-ed69-48fe-9b8e-365e69141104 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:53.467001 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69cf709-ed69-48fe-9b8e-365e69141104 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:53.478436 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1371/2725] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:17:53.483517 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89d770-ae2d-4d1f-a2e4-7586eaeb9732 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:53.496792 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89d770-ae2d-4d1f-a2e4-7586eaeb9732 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:53.502346 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8dac1e-f67d-4495-9f69-75ea1f1756e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:53.503224 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8dac1e-f67d-4495-9f69-75ea1f1756e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:53.503617 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8dac1e-f67d-4495-9f69-75ea1f1756e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:53.504249 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8dac1e-f67d-4495-9f69-75ea1f1756e9 admin admin] PATH_INFO: `/v3/projects/9bc81b11cbbf43a5b1ff393933770d7e/users/6e4bf37034b74e8794cb1698c8645005/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:53.521963 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8dac1e-f67d-4495-9f69-75ea1f1756e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bc81b11cbbf43a5b1ff393933770d7e, user_id=6e4bf37034b74e8794cb1698c8645005, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:53.525067 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8dac1e-f67d-4495-9f69-75ea1f1756e9 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:53.542182 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1355/2726] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 15:17:53 2022] PUT /identity/v3/projects/9bc81b11cbbf43a5b1ff393933770d7e/users/6e4bf37034b74e8794cb1698c8645005/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:17:53.546180 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69cf709-ed69-48fe-9b8e-365e69141104 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:53.556809 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69cf709-ed69-48fe-9b8e-365e69141104 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:53.641960 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c50c456-eb30-4c86-bd5d-3e88af7fc949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:53.643027 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50c456-eb30-4c86-bd5d-3e88af7fc949 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:53.643508 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50c456-eb30-4c86-bd5d-3e88af7fc949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:53.643957 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50c456-eb30-4c86-bd5d-3e88af7fc949 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:53.644928 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c50c456-eb30-4c86-bd5d-3e88af7fc949 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:53.646698 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c50c456-eb30-4c86-bd5d-3e88af7fc949 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:53.659125 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1372/2727] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:17:53.662924 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8dac1e-f67d-4495-9f69-75ea1f1756e9 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:53.675113 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8dac1e-f67d-4495-9f69-75ea1f1756e9 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:53.681005 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc8751-aeb6-4571-85a5-2d5d507e17cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:53.681989 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc8751-aeb6-4571-85a5-2d5d507e17cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:53.682447 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc8751-aeb6-4571-85a5-2d5d507e17cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:53.683473 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc8751-aeb6-4571-85a5-2d5d507e17cd admin admin] PATH_INFO: `/v3/projects/9bc81b11cbbf43a5b1ff393933770d7e/users/6e4bf37034b74e8794cb1698c8645005/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:53.687324 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cc8751-aeb6-4571-85a5-2d5d507e17cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bc81b11cbbf43a5b1ff393933770d7e, user_id=6e4bf37034b74e8794cb1698c8645005, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:53.691612 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cc8751-aeb6-4571-85a5-2d5d507e17cd admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:53.710977 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1356/2728] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 15:17:53 2022] PUT /identity/v3/projects/9bc81b11cbbf43a5b1ff393933770d7e/users/6e4bf37034b74e8794cb1698c8645005/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:17:53.723598 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a1be7-8e10-49fd-bb0f-75034ee1e9b1 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:53.724181 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a1be7-8e10-49fd-bb0f-75034ee1e9b1 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:53.724475 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a1be7-8e10-49fd-bb0f-75034ee1e9b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:53.755528 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-6f0a1be7-8e10-49fd-bb0f-75034ee1e9b1 None None] MFA Rules not processed for user `6e4bf37034b74e8794cb1698c8645005`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:17:53.757296 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-6f0a1be7-8e10-49fd-bb0f-75034ee1e9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:53.869349 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1373/2729] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:17:53 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 15:17:53.882422 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc8751-aeb6-4571-85a5-2d5d507e17cd admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:53.884871 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-77cc8751-aeb6-4571-85a5-2d5d507e17cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:53.990599 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc8751-aeb6-4571-85a5-2d5d507e17cd admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:53.997267 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c471eb-2391-4220-863a-a583f1087b13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:53.997929 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c471eb-2391-4220-863a-a583f1087b13 service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:53.998309 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c471eb-2391-4220-863a-a583f1087b13 service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:53.998581 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c471eb-2391-4220-863a-a583f1087b13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:54.013642 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c471eb-2391-4220-863a-a583f1087b13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:54.015571 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c471eb-2391-4220-863a-a583f1087b13 service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:54.038992 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1357/2730] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:17:53 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:17:54.178855 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a1be7-8e10-49fd-bb0f-75034ee1e9b1 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:54.192624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a1be7-8e10-49fd-bb0f-75034ee1e9b1 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:54.200709 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f038f4d7-3ff0-47d2-bb65-b4e7ffc04b02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:54.202924 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f038f4d7-3ff0-47d2-bb65-b4e7ffc04b02 service cinder] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:54.203631 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f038f4d7-3ff0-47d2-bb65-b4e7ffc04b02 service cinder] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:54.204165 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f038f4d7-3ff0-47d2-bb65-b4e7ffc04b02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:54.280263 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:54.301173 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f038f4d7-3ff0-47d2-bb65-b4e7ffc04b02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:54.302727 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f038f4d7-3ff0-47d2-bb65-b4e7ffc04b02 service cinder] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:54.317670 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1374/2731] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:17:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:17:57.136951 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c471eb-2391-4220-863a-a583f1087b13 service cinder] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:57.172000 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:57.257505 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c471eb-2391-4220-863a-a583f1087b13 service cinder] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:57.265545 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390bb5b6-8d84-4604-b555-69f94bd7c099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:57.266377 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390bb5b6-8d84-4604-b555-69f94bd7c099 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:57.266706 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390bb5b6-8d84-4604-b555-69f94bd7c099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:57.267311 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390bb5b6-8d84-4604-b555-69f94bd7c099 admin admin] PATH_INFO: `/v3/users/113348f1776e45fe8ab331aea5ab010c` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:57.268315 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390bb5b6-8d84-4604-b555-69f94bd7c099 admin admin] RBAC: Authorizing `identity:delete_user(user_id=113348f1776e45fe8ab331aea5ab010c)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:57.270598 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390bb5b6-8d84-4604-b555-69f94bd7c099 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:57.351245 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-390bb5b6-8d84-4604-b555-69f94bd7c099 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '113348f1776e45fe8ab331aea5ab010c'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:57.352309 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-390bb5b6-8d84-4604-b555-69f94bd7c099 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '113348f1776e45fe8ab331aea5ab010c'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:57.373463 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-390bb5b6-8d84-4604-b555-69f94bd7c099 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '113348f1776e45fe8ab331aea5ab010c'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:57.383396 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-390bb5b6-8d84-4604-b555-69f94bd7c099 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '113348f1776e45fe8ab331aea5ab010c'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:57.393053 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-390bb5b6-8d84-4604-b555-69f94bd7c099 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '113348f1776e45fe8ab331aea5ab010c'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:57.406653 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1358/2732] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 15:17:57 2022] DELETE /identity/v3/users/113348f1776e45fe8ab331aea5ab010c => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:17:57.411281 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58626bb-f33d-4bc0-9d1e-69c0751aeb56 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:57.411790 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58626bb-f33d-4bc0-9d1e-69c0751aeb56 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:57.412449 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58626bb-f33d-4bc0-9d1e-69c0751aeb56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:57.438525 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-d58626bb-f33d-4bc0-9d1e-69c0751aeb56 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:17:57.440740 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-d58626bb-f33d-4bc0-9d1e-69c0751aeb56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:57.524935 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1375/2733] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:17:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:17:57.535353 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390bb5b6-8d84-4604-b555-69f94bd7c099 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:57.537153 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-390bb5b6-8d84-4604-b555-69f94bd7c099 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:57.610844 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390bb5b6-8d84-4604-b555-69f94bd7c099 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:57.617066 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c9d9b0-7d70-4e32-aa59-a635d81b62f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:57.617816 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c9d9b0-7d70-4e32-aa59-a635d81b62f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:57.618166 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c9d9b0-7d70-4e32-aa59-a635d81b62f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:57.619734 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c9d9b0-7d70-4e32-aa59-a635d81b62f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:57.630437 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c9d9b0-7d70-4e32-aa59-a635d81b62f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:57.632337 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c9d9b0-7d70-4e32-aa59-a635d81b62f5 service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:57.644721 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1359/2734] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:17:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:17:57.937339 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b4cd08-68ab-4369-95f7-dd83c733eeae None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:57.937878 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b4cd08-68ab-4369-95f7-dd83c733eeae None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:57.938456 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b4cd08-68ab-4369-95f7-dd83c733eeae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:57.972827 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-80b4cd08-68ab-4369-95f7-dd83c733eeae None None] MFA Rules not processed for user `61fc09826a874f7f9a9d25238e5dceb8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:17:57.974070 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-80b4cd08-68ab-4369-95f7-dd83c733eeae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:58.012350 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c9d9b0-7d70-4e32-aa59-a635d81b62f5 service neutron] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:58.013823 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-b2c9d9b0-7d70-4e32-aa59-a635d81b62f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:58.055184 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c9d9b0-7d70-4e32-aa59-a635d81b62f5 service neutron] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:58.064412 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505748c6-f9a6-469c-8887-0d96fbe9d822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:58.065083 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505748c6-f9a6-469c-8887-0d96fbe9d822 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:58.065362 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505748c6-f9a6-469c-8887-0d96fbe9d822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:58.065631 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505748c6-f9a6-469c-8887-0d96fbe9d822 admin admin] PATH_INFO: `/v3/projects/ad481ec6814944d798441af409f5fed7` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:58.068228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505748c6-f9a6-469c-8887-0d96fbe9d822 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad481ec6814944d798441af409f5fed7)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:58.071115 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505748c6-f9a6-469c-8887-0d96fbe9d822 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:58.081900 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1376/2735] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:17:57 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:17:58.104527 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b4cd08-68ab-4369-95f7-dd83c733eeae None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:58.107733 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-80b4cd08-68ab-4369-95f7-dd83c733eeae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:58.135410 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:58.145192 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-505748c6-f9a6-469c-8887-0d96fbe9d822 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad481ec6814944d798441af409f5fed7', 'request_id': 'req-505748c6-f9a6-469c-8887-0d96fbe9d822'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:58.152876 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-505748c6-f9a6-469c-8887-0d96fbe9d822 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad481ec6814944d798441af409f5fed7', 'request_id': 'req-505748c6-f9a6-469c-8887-0d96fbe9d822'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:58.159858 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-505748c6-f9a6-469c-8887-0d96fbe9d822 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad481ec6814944d798441af409f5fed7', 'request_id': 'req-505748c6-f9a6-469c-8887-0d96fbe9d822'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:58.183481 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-505748c6-f9a6-469c-8887-0d96fbe9d822 admin admin] The token cache is being invalidate because project ad481ec6814944d798441af409f5fed7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:17:58.183811 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-505748c6-f9a6-469c-8887-0d96fbe9d822 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:58.185420 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-505748c6-f9a6-469c-8887-0d96fbe9d822 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:58.190395 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1360/2736] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 15:17:58 2022] DELETE /identity/v3/projects/ad481ec6814944d798441af409f5fed7 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:17:58.191735 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505748c6-f9a6-469c-8887-0d96fbe9d822 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:58.193237 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-505748c6-f9a6-469c-8887-0d96fbe9d822 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:58.222097 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b4cd08-68ab-4369-95f7-dd83c733eeae None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:58.241330 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:58.291761 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505748c6-f9a6-469c-8887-0d96fbe9d822 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:58.299264 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31d018-418e-4e8a-84c4-37d9cb6a0061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:58.299264 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31d018-418e-4e8a-84c4-37d9cb6a0061 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:58.299614 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31d018-418e-4e8a-84c4-37d9cb6a0061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:58.299963 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31d018-418e-4e8a-84c4-37d9cb6a0061 admin admin] PATH_INFO: `/v3/users/6e4bf37034b74e8794cb1698c8645005` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:58.302212 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b31d018-418e-4e8a-84c4-37d9cb6a0061 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e4bf37034b74e8794cb1698c8645005)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:58.304296 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b31d018-418e-4e8a-84c4-37d9cb6a0061 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:58.317719 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0456c2-52c4-46e0-91d4-4ffde4cf78e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:58.318539 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0456c2-52c4-46e0-91d4-4ffde4cf78e2 service nova] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:58.319179 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0456c2-52c4-46e0-91d4-4ffde4cf78e2 service nova] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:58.319548 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0456c2-52c4-46e0-91d4-4ffde4cf78e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:58.322349 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-ad0456c2-52c4-46e0-91d4-4ffde4cf78e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:58.357157 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-6b31d018-418e-4e8a-84c4-37d9cb6a0061 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e4bf37034b74e8794cb1698c8645005'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:58.357785 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-6b31d018-418e-4e8a-84c4-37d9cb6a0061 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e4bf37034b74e8794cb1698c8645005'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:58.397253 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-6b31d018-418e-4e8a-84c4-37d9cb6a0061 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e4bf37034b74e8794cb1698c8645005'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:58.427004 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0456c2-52c4-46e0-91d4-4ffde4cf78e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:58.429777 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0456c2-52c4-46e0-91d4-4ffde4cf78e2 service nova] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:58.431042 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-6b31d018-418e-4e8a-84c4-37d9cb6a0061 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e4bf37034b74e8794cb1698c8645005'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:58.440204 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-6b31d018-418e-4e8a-84c4-37d9cb6a0061 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e4bf37034b74e8794cb1698c8645005'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:58.442962 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1377/2737] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:17:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:17:58.447543 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1361/2738] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 15:17:58 2022] DELETE /identity/v3/users/6e4bf37034b74e8794cb1698c8645005 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:17:58.780768 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0456c2-52c4-46e0-91d4-4ffde4cf78e2 service nova] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:58.792476 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0456c2-52c4-46e0-91d4-4ffde4cf78e2 service nova] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:58.863304 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce77c7c8-49e9-466d-9eb1-68fca42491b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:58.863891 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77c7c8-49e9-466d-9eb1-68fca42491b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:58.864200 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77c7c8-49e9-466d-9eb1-68fca42491b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:58.864473 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77c7c8-49e9-466d-9eb1-68fca42491b3 admin admin] PATH_INFO: `/v3/projects/9bc81b11cbbf43a5b1ff393933770d7e` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:58.865675 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce77c7c8-49e9-466d-9eb1-68fca42491b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bc81b11cbbf43a5b1ff393933770d7e)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:58.867845 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce77c7c8-49e9-466d-9eb1-68fca42491b3 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:58.915869 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-ce77c7c8-49e9-466d-9eb1-68fca42491b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bc81b11cbbf43a5b1ff393933770d7e', 'request_id': 'req-ce77c7c8-49e9-466d-9eb1-68fca42491b3'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:58.920058 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:58.922984 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-ce77c7c8-49e9-466d-9eb1-68fca42491b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bc81b11cbbf43a5b1ff393933770d7e', 'request_id': 'req-ce77c7c8-49e9-466d-9eb1-68fca42491b3'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:58.931472 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-ce77c7c8-49e9-466d-9eb1-68fca42491b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bc81b11cbbf43a5b1ff393933770d7e', 'request_id': 'req-ce77c7c8-49e9-466d-9eb1-68fca42491b3'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:58.945351 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-ce77c7c8-49e9-466d-9eb1-68fca42491b3 admin admin] The token cache is being invalidate because project 9bc81b11cbbf43a5b1ff393933770d7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:17:58.945641 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-ce77c7c8-49e9-466d-9eb1-68fca42491b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:58.946289 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-ce77c7c8-49e9-466d-9eb1-68fca42491b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:17:58.947872 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1378/2739] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 15:17:58 2022] DELETE /identity/v3/projects/9bc81b11cbbf43a5b1ff393933770d7e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:17:58.953600 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e4d45f-adcd-4934-84a2-32edfd0db4ab None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:58.953884 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e4d45f-adcd-4934-84a2-32edfd0db4ab None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:58.954160 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e4d45f-adcd-4934-84a2-32edfd0db4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:58.978112 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-b6e4d45f-adcd-4934-84a2-32edfd0db4ab None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:17:58.979359 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-b6e4d45f-adcd-4934-84a2-32edfd0db4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:59.019525 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:59.042599 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1362/2740] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:17:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:17:59.050542 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6b34b-6f4e-4014-a522-9fff93860731 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:59.050824 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6b34b-6f4e-4014-a522-9fff93860731 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:59.051094 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6b34b-6f4e-4014-a522-9fff93860731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:59.072688 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-92d6b34b-6f4e-4014-a522-9fff93860731 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:17:59.073900 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-92d6b34b-6f4e-4014-a522-9fff93860731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:59.082940 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1379/2741] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:17:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:17:59.087695 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e4d45f-adcd-4934-84a2-32edfd0db4ab None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:59.102259 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e4d45f-adcd-4934-84a2-32edfd0db4ab None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:59.107244 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ac812-f71b-4d2a-b793-e3caa11df8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:59.107886 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ac812-f71b-4d2a-b793-e3caa11df8c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:59.108160 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ac812-f71b-4d2a-b793-e3caa11df8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:59.108429 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ac812-f71b-4d2a-b793-e3caa11df8c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:59.109081 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ac812-f71b-4d2a-b793-e3caa11df8c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:17:59.109381 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ac812-f71b-4d2a-b793-e3caa11df8c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:59.110975 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ac812-f71b-4d2a-b793-e3caa11df8c2 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:59.124607 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1363/2742] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 15:17:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:17:59.129905 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d6b34b-6f4e-4014-a522-9fff93860731 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:59.140063 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d6b34b-6f4e-4014-a522-9fff93860731 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:59.144885 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e72789-4f32-4cb0-825e-9ffc9610520b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:59.145489 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e72789-4f32-4cb0-825e-9ffc9610520b admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:59.145765 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e72789-4f32-4cb0-825e-9ffc9610520b admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:59.146034 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e72789-4f32-4cb0-825e-9ffc9610520b admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:59.146680 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e72789-4f32-4cb0-825e-9ffc9610520b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:59.148598 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e72789-4f32-4cb0-825e-9ffc9610520b admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:59.181923 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1380/2743] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 15:17:59 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:17:59.183137 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ac812-f71b-4d2a-b793-e3caa11df8c2 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:59.193928 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ac812-f71b-4d2a-b793-e3caa11df8c2 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:59.259499 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86276205-cd51-4737-950f-2f40913f4ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:59.260145 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86276205-cd51-4737-950f-2f40913f4ce6 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:59.260676 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86276205-cd51-4737-950f-2f40913f4ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:59.260790 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86276205-cd51-4737-950f-2f40913f4ce6 admin admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:59.261474 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86276205-cd51-4737-950f-2f40913f4ce6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:59.265411 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86276205-cd51-4737-950f-2f40913f4ce6 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:59.294263 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e72789-4f32-4cb0-825e-9ffc9610520b admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:59.295107 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1364/2744] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 15:17:59 2022] POST /identity/v3/users => generated 291 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:17:59.306607 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e72789-4f32-4cb0-825e-9ffc9610520b admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:59.312182 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2add4b13-39ba-44e5-9e50-8c2f62dbe906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:59.312800 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add4b13-39ba-44e5-9e50-8c2f62dbe906 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:59.313076 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add4b13-39ba-44e5-9e50-8c2f62dbe906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:59.313346 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add4b13-39ba-44e5-9e50-8c2f62dbe906 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:59.313996 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2add4b13-39ba-44e5-9e50-8c2f62dbe906 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:59.315670 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2add4b13-39ba-44e5-9e50-8c2f62dbe906 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:59.327248 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1381/2745] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:17:59.331315 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86276205-cd51-4737-950f-2f40913f4ce6 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:59.343458 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86276205-cd51-4737-950f-2f40913f4ce6 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:59.349172 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125dfbad-4b5d-45e6-a233-d05fe1b29e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:59.349776 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125dfbad-4b5d-45e6-a233-d05fe1b29e16 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:59.350059 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125dfbad-4b5d-45e6-a233-d05fe1b29e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:59.350334 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125dfbad-4b5d-45e6-a233-d05fe1b29e16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:59.350985 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125dfbad-4b5d-45e6-a233-d05fe1b29e16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:59.352625 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125dfbad-4b5d-45e6-a233-d05fe1b29e16 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:59.366781 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1365/2746] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:17:59.371095 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2add4b13-39ba-44e5-9e50-8c2f62dbe906 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:59.383489 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2add4b13-39ba-44e5-9e50-8c2f62dbe906 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:59.389104 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f21489-eafe-43b7-97a8-eee9a5c3a157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:59.389827 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f21489-eafe-43b7-97a8-eee9a5c3a157 admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:59.390105 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f21489-eafe-43b7-97a8-eee9a5c3a157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:59.390375 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f21489-eafe-43b7-97a8-eee9a5c3a157 admin admin] PATH_INFO: `/v3/projects/4f0a5f92dad84f8f81e1f3b14da8f078/users/11e23663dc26446baf167d9beb0fd38c/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:59.414563 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f21489-eafe-43b7-97a8-eee9a5c3a157 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f0a5f92dad84f8f81e1f3b14da8f078, user_id=11e23663dc26446baf167d9beb0fd38c, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:59.417917 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f21489-eafe-43b7-97a8-eee9a5c3a157 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:59.434494 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1382/2747] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 15:17:59 2022] PUT /identity/v3/projects/4f0a5f92dad84f8f81e1f3b14da8f078/users/11e23663dc26446baf167d9beb0fd38c/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:17:59.446479 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2c2d7-6bbb-454e-8e25-86f36e4ec7b3 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:59.446772 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2c2d7-6bbb-454e-8e25-86f36e4ec7b3 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:59.447050 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2c2d7-6bbb-454e-8e25-86f36e4ec7b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:59.479079 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-2ba2c2d7-6bbb-454e-8e25-86f36e4ec7b3 None None] MFA Rules not processed for user `11e23663dc26446baf167d9beb0fd38c`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:17:59.480381 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-2ba2c2d7-6bbb-454e-8e25-86f36e4ec7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:59.543091 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3721926-1516-47b7-bda2-8755ea9f9627 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:59.544276 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3721926-1516-47b7-bda2-8755ea9f9627 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:59.544805 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3721926-1516-47b7-bda2-8755ea9f9627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:59.565719 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1366/2748] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:17:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:17:59.567131 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba2c2d7-6bbb-454e-8e25-86f36e4ec7b3 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:59.567332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-d3721926-1516-47b7-bda2-8755ea9f9627 None None] MFA Rules not processed for user `b2905d19c05b4f53ba70e2bb3d165bc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:17:59.568453 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-d3721926-1516-47b7-bda2-8755ea9f9627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:17:59.619774 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:59.634006 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1383/2749] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:17:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:17:59.654408 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba2c2d7-6bbb-454e-8e25-86f36e4ec7b3 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:59.659114 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e052e7-4fdf-468c-bd93-3a0bddd887ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:59.659769 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e052e7-4fdf-468c-bd93-3a0bddd887ff admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:59.660043 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e052e7-4fdf-468c-bd93-3a0bddd887ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:59.660312 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e052e7-4fdf-468c-bd93-3a0bddd887ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:59.660949 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e052e7-4fdf-468c-bd93-3a0bddd887ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:59.662871 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e052e7-4fdf-468c-bd93-3a0bddd887ff admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:59.694730 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1367/2750] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 15:17:59 2022] POST /identity/v3/projects => generated 379 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:17:59.698909 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3721926-1516-47b7-bda2-8755ea9f9627 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:59.717546 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3721926-1516-47b7-bda2-8755ea9f9627 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:59.802131 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d010de96-e74d-4d2e-aac0-f173d19d0866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:59.802744 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010de96-e74d-4d2e-aac0-f173d19d0866 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:59.803020 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010de96-e74d-4d2e-aac0-f173d19d0866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:59.803320 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010de96-e74d-4d2e-aac0-f173d19d0866 admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:59.803996 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d010de96-e74d-4d2e-aac0-f173d19d0866 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:59.805988 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d010de96-e74d-4d2e-aac0-f173d19d0866 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:59.835400 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1384/2751] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 15:17:59 2022] POST /identity/v3/users => generated 296 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:17:59.838706 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e052e7-4fdf-468c-bd93-3a0bddd887ff admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:59.854899 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e052e7-4fdf-468c-bd93-3a0bddd887ff admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:59.863029 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a5665-4a06-42ad-9edd-d6c402be5aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:59.864291 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a5665-4a06-42ad-9edd-d6c402be5aa4 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:59.864776 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a5665-4a06-42ad-9edd-d6c402be5aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:59.865250 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a5665-4a06-42ad-9edd-d6c402be5aa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:59.866481 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748a5665-4a06-42ad-9edd-d6c402be5aa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:59.868791 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748a5665-4a06-42ad-9edd-d6c402be5aa4 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:59.889775 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1368/2752] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 15:17:59.891347 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d010de96-e74d-4d2e-aac0-f173d19d0866 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:59.904305 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d010de96-e74d-4d2e-aac0-f173d19d0866 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:17:59.909649 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c506c1d-f184-4755-87d0-c86f97e1fdcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:17:59.910325 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c506c1d-f184-4755-87d0-c86f97e1fdcc admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:17:59.910612 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c506c1d-f184-4755-87d0-c86f97e1fdcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:17:59.910886 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c506c1d-f184-4755-87d0-c86f97e1fdcc admin admin] PATH_INFO: `/v3/projects/8396e4f7efc44544a7583f8f90c95d07/users/5c015c15ba464faf8652542e78a056d2/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:17:59.926029 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:DELETE=2,keystone:SELECT=92,keystone:INSERT=6 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:17:59.937648 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c506c1d-f184-4755-87d0-c86f97e1fdcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=8396e4f7efc44544a7583f8f90c95d07, user_id=5c015c15ba464faf8652542e78a056d2, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:17:59.940943 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c506c1d-f184-4755-87d0-c86f97e1fdcc admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:17:59.957535 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1385/2753] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 15:17:59 2022] PUT /identity/v3/projects/8396e4f7efc44544a7583f8f90c95d07/users/5c015c15ba464faf8652542e78a056d2/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:17:59.961642 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a5665-4a06-42ad-9edd-d6c402be5aa4 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:17:59.979582 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a5665-4a06-42ad-9edd-d6c402be5aa4 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:00.068092 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830e96b2-a294-4bec-9bc4-5b0e484a9e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:00.068823 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e96b2-a294-4bec-9bc4-5b0e484a9e23 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:00.069110 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e96b2-a294-4bec-9bc4-5b0e484a9e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:00.069382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e96b2-a294-4bec-9bc4-5b0e484a9e23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:00.070036 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830e96b2-a294-4bec-9bc4-5b0e484a9e23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:00.071923 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830e96b2-a294-4bec-9bc4-5b0e484a9e23 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:00.086741 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1369/2754] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:18:00.087975 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c506c1d-f184-4755-87d0-c86f97e1fdcc admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:00.102649 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c506c1d-f184-4755-87d0-c86f97e1fdcc admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:00.109955 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2382ea62-a71f-42c7-86cd-931bb6af80f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:00.110635 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382ea62-a71f-42c7-86cd-931bb6af80f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:00.110916 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382ea62-a71f-42c7-86cd-931bb6af80f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:00.111189 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382ea62-a71f-42c7-86cd-931bb6af80f7 admin admin] PATH_INFO: `/v3/projects/8396e4f7efc44544a7583f8f90c95d07/users/5c015c15ba464faf8652542e78a056d2/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:00.113233 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2382ea62-a71f-42c7-86cd-931bb6af80f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8396e4f7efc44544a7583f8f90c95d07, user_id=5c015c15ba464faf8652542e78a056d2, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:00.115954 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2382ea62-a71f-42c7-86cd-931bb6af80f7 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:00.129974 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1386/2755] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 15:18:00 2022] PUT /identity/v3/projects/8396e4f7efc44544a7583f8f90c95d07/users/5c015c15ba464faf8652542e78a056d2/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:18:00.140976 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7069e0b-36f3-4b38-a38b-081a036e1219 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:00.141200 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7069e0b-36f3-4b38-a38b-081a036e1219 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:00.141478 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7069e0b-36f3-4b38-a38b-081a036e1219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:00.218499 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-d7069e0b-36f3-4b38-a38b-081a036e1219 None None] MFA Rules not processed for user `5c015c15ba464faf8652542e78a056d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:18:00.219798 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-d7069e0b-36f3-4b38-a38b-081a036e1219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:18:00.342371 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1370/2756] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:18:00 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:18:00.354376 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2382ea62-a71f-42c7-86cd-931bb6af80f7 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:00.357437 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-2382ea62-a71f-42c7-86cd-931bb6af80f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:18:00.424549 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2382ea62-a71f-42c7-86cd-931bb6af80f7 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:00.432472 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39463b3-ff76-4df2-9ab8-244adc991d4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:00.433259 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39463b3-ff76-4df2-9ab8-244adc991d4c service cinder] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:00.433632 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39463b3-ff76-4df2-9ab8-244adc991d4c service cinder] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:00.434548 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39463b3-ff76-4df2-9ab8-244adc991d4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:00.455831 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39463b3-ff76-4df2-9ab8-244adc991d4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:00.457353 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39463b3-ff76-4df2-9ab8-244adc991d4c service cinder] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:00.469601 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1387/2757] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:18:00 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:18:00.885634 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7069e0b-36f3-4b38-a38b-081a036e1219 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:00.920382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:18:00.951778 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7069e0b-36f3-4b38-a38b-081a036e1219 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:00.956532 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80618674-90ea-463e-a88a-6f9469e652cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:00.957181 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80618674-90ea-463e-a88a-6f9469e652cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:00.957502 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80618674-90ea-463e-a88a-6f9469e652cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:00.957736 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80618674-90ea-463e-a88a-6f9469e652cc admin admin] PATH_INFO: `/v3/users/11e23663dc26446baf167d9beb0fd38c` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:00.958876 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80618674-90ea-463e-a88a-6f9469e652cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=11e23663dc26446baf167d9beb0fd38c)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:00.960831 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80618674-90ea-463e-a88a-6f9469e652cc admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:01.009491 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-80618674-90ea-463e-a88a-6f9469e652cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11e23663dc26446baf167d9beb0fd38c'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:01.010165 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-80618674-90ea-463e-a88a-6f9469e652cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11e23663dc26446baf167d9beb0fd38c'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:01.027785 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-80618674-90ea-463e-a88a-6f9469e652cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11e23663dc26446baf167d9beb0fd38c'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:01.039977 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-80618674-90ea-463e-a88a-6f9469e652cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11e23663dc26446baf167d9beb0fd38c'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:01.047882 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-80618674-90ea-463e-a88a-6f9469e652cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11e23663dc26446baf167d9beb0fd38c'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:01.054457 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1371/2758] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 15:18:00 2022] DELETE /identity/v3/users/11e23663dc26446baf167d9beb0fd38c => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:18:01.059548 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0deba-a4d4-4472-96c8-ce3e11b68dd3 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:01.060043 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0deba-a4d4-4472-96c8-ce3e11b68dd3 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:01.060322 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0deba-a4d4-4472-96c8-ce3e11b68dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:01.079171 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-71b0deba-a4d4-4472-96c8-ce3e11b68dd3 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:18:01.080389 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-71b0deba-a4d4-4472-96c8-ce3e11b68dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:18:01.145759 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1388/2759] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:18:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:18:01.155871 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80618674-90ea-463e-a88a-6f9469e652cc admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:01.157287 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-80618674-90ea-463e-a88a-6f9469e652cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:18:01.218772 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80618674-90ea-463e-a88a-6f9469e652cc admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:01.222679 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b292e324-17a6-4b7e-b575-10c4a22b2ec8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:01.223335 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b292e324-17a6-4b7e-b575-10c4a22b2ec8 service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:01.223621 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b292e324-17a6-4b7e-b575-10c4a22b2ec8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:01.223898 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b292e324-17a6-4b7e-b575-10c4a22b2ec8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:01.234046 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b292e324-17a6-4b7e-b575-10c4a22b2ec8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:01.235517 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b292e324-17a6-4b7e-b575-10c4a22b2ec8 service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:01.246885 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1372/2760] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:18:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:18:01.544822 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0deba-a4d4-4472-96c8-ce3e11b68dd3 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:01.546424 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-71b0deba-a4d4-4472-96c8-ce3e11b68dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:18:01.579530 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0deba-a4d4-4472-96c8-ce3e11b68dd3 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:01.584270 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8174c6a-4405-4a73-8173-eb596ac96cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:01.584907 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8174c6a-4405-4a73-8173-eb596ac96cfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:01.585182 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8174c6a-4405-4a73-8173-eb596ac96cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:01.585445 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8174c6a-4405-4a73-8173-eb596ac96cfe admin admin] PATH_INFO: `/v3/projects/4f0a5f92dad84f8f81e1f3b14da8f078` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:01.586583 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8174c6a-4405-4a73-8173-eb596ac96cfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f0a5f92dad84f8f81e1f3b14da8f078)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:01.588506 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8174c6a-4405-4a73-8173-eb596ac96cfe admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:01.643670 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-a8174c6a-4405-4a73-8173-eb596ac96cfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f0a5f92dad84f8f81e1f3b14da8f078', 'request_id': 'req-a8174c6a-4405-4a73-8173-eb596ac96cfe'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:01.649969 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-a8174c6a-4405-4a73-8173-eb596ac96cfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f0a5f92dad84f8f81e1f3b14da8f078', 'request_id': 'req-a8174c6a-4405-4a73-8173-eb596ac96cfe'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:01.658225 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-a8174c6a-4405-4a73-8173-eb596ac96cfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f0a5f92dad84f8f81e1f3b14da8f078', 'request_id': 'req-a8174c6a-4405-4a73-8173-eb596ac96cfe'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:01.672533 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-a8174c6a-4405-4a73-8173-eb596ac96cfe admin admin] The token cache is being invalidate because project 4f0a5f92dad84f8f81e1f3b14da8f078 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:18:01.672854 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-a8174c6a-4405-4a73-8173-eb596ac96cfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:01.673603 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-a8174c6a-4405-4a73-8173-eb596ac96cfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:01.675167 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1389/2761] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 15:18:01 2022] DELETE /identity/v3/projects/4f0a5f92dad84f8f81e1f3b14da8f078 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:18:01.679344 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b292e324-17a6-4b7e-b575-10c4a22b2ec8 service neutron] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:01.689352 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b292e324-17a6-4b7e-b575-10c4a22b2ec8 service neutron] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:01.746335 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:18:01.754557 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b820039d-2c67-4791-990c-ef5674537b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:01.755651 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b820039d-2c67-4791-990c-ef5674537b10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:01.756119 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b820039d-2c67-4791-990c-ef5674537b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:01.756568 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b820039d-2c67-4791-990c-ef5674537b10 admin admin] PATH_INFO: `/v3/users/5c015c15ba464faf8652542e78a056d2` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:01.757765 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b820039d-2c67-4791-990c-ef5674537b10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c015c15ba464faf8652542e78a056d2)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:01.760131 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b820039d-2c67-4791-990c-ef5674537b10 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:01.814465 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b820039d-2c67-4791-990c-ef5674537b10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c015c15ba464faf8652542e78a056d2'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:01.815581 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b820039d-2c67-4791-990c-ef5674537b10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c015c15ba464faf8652542e78a056d2'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:01.828629 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b820039d-2c67-4791-990c-ef5674537b10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c015c15ba464faf8652542e78a056d2'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:01.835597 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b820039d-2c67-4791-990c-ef5674537b10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c015c15ba464faf8652542e78a056d2'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:01.844110 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b820039d-2c67-4791-990c-ef5674537b10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c015c15ba464faf8652542e78a056d2'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:01.850542 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1373/2762] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 15:18:01 2022] DELETE /identity/v3/users/5c015c15ba464faf8652542e78a056d2 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:18:02.179892 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8174c6a-4405-4a73-8173-eb596ac96cfe admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:02.181545 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-a8174c6a-4405-4a73-8173-eb596ac96cfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:18:02.190685 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:18:02.203833 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8174c6a-4405-4a73-8173-eb596ac96cfe admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:02.273967 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb44db96-4b2c-4abd-b82a-7ed2f2bd2da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:02.274686 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb44db96-4b2c-4abd-b82a-7ed2f2bd2da4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:02.274953 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb44db96-4b2c-4abd-b82a-7ed2f2bd2da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:02.275254 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb44db96-4b2c-4abd-b82a-7ed2f2bd2da4 admin admin] PATH_INFO: `/v3/projects/8396e4f7efc44544a7583f8f90c95d07` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:02.276724 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb44db96-4b2c-4abd-b82a-7ed2f2bd2da4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8396e4f7efc44544a7583f8f90c95d07)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:02.279119 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb44db96-4b2c-4abd-b82a-7ed2f2bd2da4 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:02.339229 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-bb44db96-4b2c-4abd-b82a-7ed2f2bd2da4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8396e4f7efc44544a7583f8f90c95d07', 'request_id': 'req-bb44db96-4b2c-4abd-b82a-7ed2f2bd2da4'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:02.345315 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-bb44db96-4b2c-4abd-b82a-7ed2f2bd2da4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8396e4f7efc44544a7583f8f90c95d07', 'request_id': 'req-bb44db96-4b2c-4abd-b82a-7ed2f2bd2da4'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:02.352251 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-bb44db96-4b2c-4abd-b82a-7ed2f2bd2da4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8396e4f7efc44544a7583f8f90c95d07', 'request_id': 'req-bb44db96-4b2c-4abd-b82a-7ed2f2bd2da4'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:02.376116 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-bb44db96-4b2c-4abd-b82a-7ed2f2bd2da4 admin admin] The token cache is being invalidate because project 8396e4f7efc44544a7583f8f90c95d07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:18:02.376869 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-bb44db96-4b2c-4abd-b82a-7ed2f2bd2da4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:02.378167 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-bb44db96-4b2c-4abd-b82a-7ed2f2bd2da4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:02.382920 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1390/2763] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 15:18:02 2022] DELETE /identity/v3/projects/8396e4f7efc44544a7583f8f90c95d07 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:18:02.385698 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475b744-97c3-4305-85b2-b8e5a2ef2480 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:02.386147 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475b744-97c3-4305-85b2-b8e5a2ef2480 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:02.386382 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475b744-97c3-4305-85b2-b8e5a2ef2480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:02.414978 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-4475b744-97c3-4305-85b2-b8e5a2ef2480 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:18:02.416908 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-4475b744-97c3-4305-85b2-b8e5a2ef2480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:18:02.493465 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1374/2764] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:18:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:18:02.501328 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc73cd-9822-40cb-bd6f-023bfb7a4d69 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:02.501757 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc73cd-9822-40cb-bd6f-023bfb7a4d69 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:02.502212 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc73cd-9822-40cb-bd6f-023bfb7a4d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:02.522609 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-29cc73cd-9822-40cb-bd6f-023bfb7a4d69 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:18:02.523847 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-29cc73cd-9822-40cb-bd6f-023bfb7a4d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:18:02.535245 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1391/2765] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:18:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:18:02.536528 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4475b744-97c3-4305-85b2-b8e5a2ef2480 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:02.548133 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4475b744-97c3-4305-85b2-b8e5a2ef2480 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:02.560102 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ecfd3-e92e-493f-979a-52b86935cde1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:02.561621 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ecfd3-e92e-493f-979a-52b86935cde1 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:02.561621 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ecfd3-e92e-493f-979a-52b86935cde1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:02.561621 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ecfd3-e92e-493f-979a-52b86935cde1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:02.562061 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85ecfd3-e92e-493f-979a-52b86935cde1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:18:02.562268 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85ecfd3-e92e-493f-979a-52b86935cde1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:02.564363 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85ecfd3-e92e-493f-979a-52b86935cde1 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:02.579050 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1375/2766] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 15:18:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:18:02.583421 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cc73cd-9822-40cb-bd6f-023bfb7a4d69 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:02.593938 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cc73cd-9822-40cb-bd6f-023bfb7a4d69 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:02.599081 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16909bb-f2e4-4df8-93d5-5175a08853ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:02.599804 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16909bb-f2e4-4df8-93d5-5175a08853ea admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:02.600082 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16909bb-f2e4-4df8-93d5-5175a08853ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:02.600355 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16909bb-f2e4-4df8-93d5-5175a08853ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:02.601014 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16909bb-f2e4-4df8-93d5-5175a08853ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:02.603080 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16909bb-f2e4-4df8-93d5-5175a08853ea admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:02.636819 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1392/2767] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 15:18:02 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:18:02.640784 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ecfd3-e92e-493f-979a-52b86935cde1 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:02.652015 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ecfd3-e92e-493f-979a-52b86935cde1 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:02.727133 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbe8f27-230a-4bc5-9a55-7e7e25f31bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:02.727872 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe8f27-230a-4bc5-9a55-7e7e25f31bbc admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:02.728158 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe8f27-230a-4bc5-9a55-7e7e25f31bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:02.728433 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe8f27-230a-4bc5-9a55-7e7e25f31bbc admin admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:02.729226 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbe8f27-230a-4bc5-9a55-7e7e25f31bbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:02.731870 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbe8f27-230a-4bc5-9a55-7e7e25f31bbc admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:02.744896 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:18:02.756187 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1376/2768] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 15:18:02 2022] POST /identity/v3/users => generated 293 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:18:02.760438 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16909bb-f2e4-4df8-93d5-5175a08853ea admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:02.773341 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16909bb-f2e4-4df8-93d5-5175a08853ea admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:02.782142 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e835c7-f83f-4657-b87d-0cfb0a227c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:02.782782 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e835c7-f83f-4657-b87d-0cfb0a227c77 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:02.783063 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e835c7-f83f-4657-b87d-0cfb0a227c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:02.783368 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e835c7-f83f-4657-b87d-0cfb0a227c77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:02.784040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e835c7-f83f-4657-b87d-0cfb0a227c77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:02.785716 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e835c7-f83f-4657-b87d-0cfb0a227c77 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:02.797767 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1393/2769] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:18:02.801724 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbe8f27-230a-4bc5-9a55-7e7e25f31bbc admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:02.812652 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbe8f27-230a-4bc5-9a55-7e7e25f31bbc admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:02.817528 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99d9696-92a2-482b-b90b-24040895811a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:02.818139 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d9696-92a2-482b-b90b-24040895811a admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:02.818417 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d9696-92a2-482b-b90b-24040895811a admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:02.818687 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d9696-92a2-482b-b90b-24040895811a admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:02.819464 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99d9696-92a2-482b-b90b-24040895811a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:02.821106 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99d9696-92a2-482b-b90b-24040895811a admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:02.833637 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1377/2770] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:18:02.837638 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e835c7-f83f-4657-b87d-0cfb0a227c77 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:02.848892 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e835c7-f83f-4657-b87d-0cfb0a227c77 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:02.853667 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc61fad-2eaf-4427-9752-8999f3595963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:02.854336 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc61fad-2eaf-4427-9752-8999f3595963 admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:02.854622 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc61fad-2eaf-4427-9752-8999f3595963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:02.854893 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc61fad-2eaf-4427-9752-8999f3595963 admin admin] PATH_INFO: `/v3/projects/c0dc4f5e90634557a18ca4d90df9bea2/users/a0222a390a404355ac7a03fea40de468/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:02.873295 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc61fad-2eaf-4427-9752-8999f3595963 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0dc4f5e90634557a18ca4d90df9bea2, user_id=a0222a390a404355ac7a03fea40de468, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:02.876595 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc61fad-2eaf-4427-9752-8999f3595963 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:02.891876 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1394/2771] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 15:18:02 2022] PUT /identity/v3/projects/c0dc4f5e90634557a18ca4d90df9bea2/users/a0222a390a404355ac7a03fea40de468/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:18:02.904176 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b5821-6068-4925-8c4b-7f6841b25369 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:02.904549 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b5821-6068-4925-8c4b-7f6841b25369 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:02.904892 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b5821-6068-4925-8c4b-7f6841b25369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:02.935040 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-b87b5821-6068-4925-8c4b-7f6841b25369 None None] MFA Rules not processed for user `a0222a390a404355ac7a03fea40de468`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:18:02.936333 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-b87b5821-6068-4925-8c4b-7f6841b25369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:18:03.008391 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1378/2772] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:18:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:18:03.019325 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc61fad-2eaf-4427-9752-8999f3595963 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:03.021045 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-5fc61fad-2eaf-4427-9752-8999f3595963 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:18:03.055227 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:18:03.091729 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc61fad-2eaf-4427-9752-8999f3595963 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:03.095700 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff155c8a-bb54-4e26-984d-8594c6532900 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:03.096591 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff155c8a-bb54-4e26-984d-8594c6532900 service cinder] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:03.096880 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff155c8a-bb54-4e26-984d-8594c6532900 service cinder] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:03.097654 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff155c8a-bb54-4e26-984d-8594c6532900 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:03.112068 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff155c8a-bb54-4e26-984d-8594c6532900 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:03.113670 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff155c8a-bb54-4e26-984d-8594c6532900 service cinder] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:03.123576 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1395/2773] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:18:03 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:18:12.596549 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87b5821-6068-4925-8c4b-7f6841b25369 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:12.658111 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87b5821-6068-4925-8c4b-7f6841b25369 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:12.662545 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94896ae-bc4e-47b9-ae8c-821a1574cee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:12.663157 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94896ae-bc4e-47b9-ae8c-821a1574cee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:12.663483 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94896ae-bc4e-47b9-ae8c-821a1574cee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:12.663754 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94896ae-bc4e-47b9-ae8c-821a1574cee3 admin admin] PATH_INFO: `/v3/users/a0222a390a404355ac7a03fea40de468` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:12.664868 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94896ae-bc4e-47b9-ae8c-821a1574cee3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0222a390a404355ac7a03fea40de468)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:12.666800 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94896ae-bc4e-47b9-ae8c-821a1574cee3 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:12.712182 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-a94896ae-bc4e-47b9-ae8c-821a1574cee3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0222a390a404355ac7a03fea40de468'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:12.712846 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-a94896ae-bc4e-47b9-ae8c-821a1574cee3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0222a390a404355ac7a03fea40de468'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:12.725836 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-a94896ae-bc4e-47b9-ae8c-821a1574cee3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0222a390a404355ac7a03fea40de468'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:12.731150 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:INSERT=2,keystone:SELECT=89,keystone:DELETE=9 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:18:12.733177 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-a94896ae-bc4e-47b9-ae8c-821a1574cee3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0222a390a404355ac7a03fea40de468'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:12.740878 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-a94896ae-bc4e-47b9-ae8c-821a1574cee3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0222a390a404355ac7a03fea40de468'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:12.747831 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1379/2774] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 15:18:12 2022] DELETE /identity/v3/users/a0222a390a404355ac7a03fea40de468 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:18:12.752355 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f58cf1-b7d0-441c-b26f-08bb0d48d799 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:12.752874 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f58cf1-b7d0-441c-b26f-08bb0d48d799 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:12.752983 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f58cf1-b7d0-441c-b26f-08bb0d48d799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:12.772786 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-48f58cf1-b7d0-441c-b26f-08bb0d48d799 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:18:12.774093 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-48f58cf1-b7d0-441c-b26f-08bb0d48d799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:18:12.835061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1396/2775] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:18:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:18:12.845475 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94896ae-bc4e-47b9-ae8c-821a1574cee3 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:12.846877 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-a94896ae-bc4e-47b9-ae8c-821a1574cee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:18:12.906403 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94896ae-bc4e-47b9-ae8c-821a1574cee3 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:12.910263 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b387d-56c5-41da-b1c0-eda05cc8dd77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:12.910809 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b387d-56c5-41da-b1c0-eda05cc8dd77 service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:12.911092 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b387d-56c5-41da-b1c0-eda05cc8dd77 service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:12.911415 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b387d-56c5-41da-b1c0-eda05cc8dd77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:12.921118 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9b387d-56c5-41da-b1c0-eda05cc8dd77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:12.922787 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9b387d-56c5-41da-b1c0-eda05cc8dd77 service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:12.932639 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1380/2776] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:18:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:18:13.197060 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f58cf1-b7d0-441c-b26f-08bb0d48d799 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:13.198476 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-48f58cf1-b7d0-441c-b26f-08bb0d48d799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:18:13.211662 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f58cf1-b7d0-441c-b26f-08bb0d48d799 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:13.217337 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac05957-7d02-4ed1-853d-e853078a06fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:13.218088 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac05957-7d02-4ed1-853d-e853078a06fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:13.218431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac05957-7d02-4ed1-853d-e853078a06fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:13.218736 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac05957-7d02-4ed1-853d-e853078a06fd admin admin] PATH_INFO: `/v3/projects/c0dc4f5e90634557a18ca4d90df9bea2` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:13.219880 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac05957-7d02-4ed1-853d-e853078a06fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0dc4f5e90634557a18ca4d90df9bea2)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:13.221781 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac05957-7d02-4ed1-853d-e853078a06fd admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:13.269825 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-cac05957-7d02-4ed1-853d-e853078a06fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0dc4f5e90634557a18ca4d90df9bea2', 'request_id': 'req-cac05957-7d02-4ed1-853d-e853078a06fd'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:13.275171 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-cac05957-7d02-4ed1-853d-e853078a06fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0dc4f5e90634557a18ca4d90df9bea2', 'request_id': 'req-cac05957-7d02-4ed1-853d-e853078a06fd'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:13.281436 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-cac05957-7d02-4ed1-853d-e853078a06fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0dc4f5e90634557a18ca4d90df9bea2', 'request_id': 'req-cac05957-7d02-4ed1-853d-e853078a06fd'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:13.294075 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-cac05957-7d02-4ed1-853d-e853078a06fd admin admin] The token cache is being invalidate because project c0dc4f5e90634557a18ca4d90df9bea2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:18:13.294374 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-cac05957-7d02-4ed1-853d-e853078a06fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:13.295003 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-cac05957-7d02-4ed1-853d-e853078a06fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:13.296520 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1397/2777] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 15:18:13 2022] DELETE /identity/v3/projects/c0dc4f5e90634557a18ca4d90df9bea2 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:18:13.301349 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f805121-dd66-4659-9d7b-e82dcdb26e25 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:13.301633 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f805121-dd66-4659-9d7b-e82dcdb26e25 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:13.301904 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f805121-dd66-4659-9d7b-e82dcdb26e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:13.322835 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-7f805121-dd66-4659-9d7b-e82dcdb26e25 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:18:13.324967 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-7f805121-dd66-4659-9d7b-e82dcdb26e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:18:13.384217 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1381/2778] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:18:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:18:13.392556 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476fa5c-1c03-4cc7-9a65-0a2a648cf0ca None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:13.392851 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476fa5c-1c03-4cc7-9a65-0a2a648cf0ca None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:13.393129 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476fa5c-1c03-4cc7-9a65-0a2a648cf0ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:13.413058 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-c476fa5c-1c03-4cc7-9a65-0a2a648cf0ca None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:18:13.414295 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-c476fa5c-1c03-4cc7-9a65-0a2a648cf0ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:18:13.422889 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1398/2779] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:18:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 15:18:13.426956 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f805121-dd66-4659-9d7b-e82dcdb26e25 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:13.438084 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f805121-dd66-4659-9d7b-e82dcdb26e25 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:13.442449 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc7ac4-78a8-46ad-979a-1dec9ee9b287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:13.442974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc7ac4-78a8-46ad-979a-1dec9ee9b287 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:13.443284 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc7ac4-78a8-46ad-979a-1dec9ee9b287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:13.443563 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc7ac4-78a8-46ad-979a-1dec9ee9b287 admin admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:13.444210 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dc7ac4-78a8-46ad-979a-1dec9ee9b287 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:18:13.444512 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dc7ac4-78a8-46ad-979a-1dec9ee9b287 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:13.446104 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dc7ac4-78a8-46ad-979a-1dec9ee9b287 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:13.457697 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1382/2780] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 15:18:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:18:13.462030 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c476fa5c-1c03-4cc7-9a65-0a2a648cf0ca None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:13.471288 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c476fa5c-1c03-4cc7-9a65-0a2a648cf0ca None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:13.475980 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700bc240-a0d2-4de7-ba66-9f9572ad6027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:13.476575 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700bc240-a0d2-4de7-ba66-9f9572ad6027 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:13.476855 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700bc240-a0d2-4de7-ba66-9f9572ad6027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:13.477125 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700bc240-a0d2-4de7-ba66-9f9572ad6027 admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:13.477753 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700bc240-a0d2-4de7-ba66-9f9572ad6027 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:13.479682 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700bc240-a0d2-4de7-ba66-9f9572ad6027 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:13.508218 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1399/2781] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 15:18:13 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:18:13.512444 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc7ac4-78a8-46ad-979a-1dec9ee9b287 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:13.522818 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc7ac4-78a8-46ad-979a-1dec9ee9b287 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:13.580561 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e09ab92-4620-48a2-8624-3cea43b6b7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:13.581179 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e09ab92-4620-48a2-8624-3cea43b6b7d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:13.581441 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e09ab92-4620-48a2-8624-3cea43b6b7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:13.581705 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e09ab92-4620-48a2-8624-3cea43b6b7d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:13.582371 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e09ab92-4620-48a2-8624-3cea43b6b7d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:13.584293 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e09ab92-4620-48a2-8624-3cea43b6b7d8 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:13.605715 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1383/2782] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 15:18:13 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:18:13.609861 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700bc240-a0d2-4de7-ba66-9f9572ad6027 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:13.620600 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700bc240-a0d2-4de7-ba66-9f9572ad6027 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:13.626115 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ccff7-9b81-4eec-84c7-8cc305b621ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:13.626719 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ccff7-9b81-4eec-84c7-8cc305b621ff admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:13.627000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ccff7-9b81-4eec-84c7-8cc305b621ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:13.627318 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ccff7-9b81-4eec-84c7-8cc305b621ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:13.627966 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385ccff7-9b81-4eec-84c7-8cc305b621ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:13.629590 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385ccff7-9b81-4eec-84c7-8cc305b621ff admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:13.639720 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1400/2783] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:18:13.643704 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e09ab92-4620-48a2-8624-3cea43b6b7d8 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:13.654165 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e09ab92-4620-48a2-8624-3cea43b6b7d8 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:13.658806 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b441bb-b226-48f0-b46d-289de2cabe1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:13.659433 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b441bb-b226-48f0-b46d-289de2cabe1e admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:13.659718 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b441bb-b226-48f0-b46d-289de2cabe1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:13.660011 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b441bb-b226-48f0-b46d-289de2cabe1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:13.660655 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b441bb-b226-48f0-b46d-289de2cabe1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:13.662269 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b441bb-b226-48f0-b46d-289de2cabe1e admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:13.664327 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:18:13.674596 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1384/2784] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:18:13.678570 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ccff7-9b81-4eec-84c7-8cc305b621ff admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:13.688554 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ccff7-9b81-4eec-84c7-8cc305b621ff admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:13.693275 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a825b-7d28-4344-b85a-e54cb8bd3171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:13.693857 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a825b-7d28-4344-b85a-e54cb8bd3171 admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:13.694136 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a825b-7d28-4344-b85a-e54cb8bd3171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:13.694408 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a825b-7d28-4344-b85a-e54cb8bd3171 admin admin] PATH_INFO: `/v3/projects/896c1d3b2d664d19bf6afc0f8bd19ed7/users/72842b1d24194827883e8c223516e6a9/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:13.706283 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:18:13.709613 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20a825b-7d28-4344-b85a-e54cb8bd3171 admin admin] RBAC: Authorizing `identity:create_grant(project_id=896c1d3b2d664d19bf6afc0f8bd19ed7, user_id=72842b1d24194827883e8c223516e6a9, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:13.712491 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20a825b-7d28-4344-b85a-e54cb8bd3171 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:13.729609 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1401/2785] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 15:18:13 2022] PUT /identity/v3/projects/896c1d3b2d664d19bf6afc0f8bd19ed7/users/72842b1d24194827883e8c223516e6a9/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:18:13.744503 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6f1cf-91f2-440b-8f3c-723adb106b05 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:13.744793 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6f1cf-91f2-440b-8f3c-723adb106b05 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:13.745065 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6f1cf-91f2-440b-8f3c-723adb106b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:13.772704 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-b5d6f1cf-91f2-440b-8f3c-723adb106b05 None None] MFA Rules not processed for user `72842b1d24194827883e8c223516e6a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:18:13.773952 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-b5d6f1cf-91f2-440b-8f3c-723adb106b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:18:13.838649 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1385/2786] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:18:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:18:13.850263 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a825b-7d28-4344-b85a-e54cb8bd3171 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:13.852044 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-a20a825b-7d28-4344-b85a-e54cb8bd3171 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:18:13.909372 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a825b-7d28-4344-b85a-e54cb8bd3171 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:13.913075 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10914305-00c7-4efc-8d5a-580c8451c6f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:13.913664 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10914305-00c7-4efc-8d5a-580c8451c6f6 service cinder] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:13.913945 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10914305-00c7-4efc-8d5a-580c8451c6f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:13.914218 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10914305-00c7-4efc-8d5a-580c8451c6f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:13.922944 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10914305-00c7-4efc-8d5a-580c8451c6f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:13.924409 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10914305-00c7-4efc-8d5a-580c8451c6f6 service cinder] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:13.933097 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1402/2787] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:18:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:18:13.957546 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d6f1cf-91f2-440b-8f3c-723adb106b05 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:14.021018 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d6f1cf-91f2-440b-8f3c-723adb106b05 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:14.025589 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40074073-b05d-4c00-9dd3-8b6745060588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:14.026207 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40074073-b05d-4c00-9dd3-8b6745060588 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:14.026481 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40074073-b05d-4c00-9dd3-8b6745060588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:14.026750 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40074073-b05d-4c00-9dd3-8b6745060588 admin admin] PATH_INFO: `/v3/users/72842b1d24194827883e8c223516e6a9` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:14.029031 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40074073-b05d-4c00-9dd3-8b6745060588 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72842b1d24194827883e8c223516e6a9)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:14.030916 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40074073-b05d-4c00-9dd3-8b6745060588 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:14.077929 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-40074073-b05d-4c00-9dd3-8b6745060588 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72842b1d24194827883e8c223516e6a9'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:14.078596 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-40074073-b05d-4c00-9dd3-8b6745060588 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72842b1d24194827883e8c223516e6a9'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:14.091110 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-40074073-b05d-4c00-9dd3-8b6745060588 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72842b1d24194827883e8c223516e6a9'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:14.097782 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-40074073-b05d-4c00-9dd3-8b6745060588 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72842b1d24194827883e8c223516e6a9'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:14.105231 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-40074073-b05d-4c00-9dd3-8b6745060588 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72842b1d24194827883e8c223516e6a9'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:14.111331 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1386/2788] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 15:18:13 2022] DELETE /identity/v3/users/72842b1d24194827883e8c223516e6a9 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:18:14.116065 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b69bd56-7a7e-4136-ad29-210d13fdbaa1 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:14.116382 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b69bd56-7a7e-4136-ad29-210d13fdbaa1 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:14.116655 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b69bd56-7a7e-4136-ad29-210d13fdbaa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:14.138089 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-0b69bd56-7a7e-4136-ad29-210d13fdbaa1 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:18:14.139281 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-0b69bd56-7a7e-4136-ad29-210d13fdbaa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:18:14.196483 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1403/2789] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:18:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:18:14.207099 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40074073-b05d-4c00-9dd3-8b6745060588 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:14.208559 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-40074073-b05d-4c00-9dd3-8b6745060588 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:18:14.219169 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:18:14.267682 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40074073-b05d-4c00-9dd3-8b6745060588 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:14.271366 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5487a447-c353-4033-b38f-92955f777de4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:14.271966 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5487a447-c353-4033-b38f-92955f777de4 service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:14.272245 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5487a447-c353-4033-b38f-92955f777de4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:14.272514 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5487a447-c353-4033-b38f-92955f777de4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:14.281899 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5487a447-c353-4033-b38f-92955f777de4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:14.283349 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5487a447-c353-4033-b38f-92955f777de4 service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:14.292686 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1387/2790] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:18:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:18:14.552432 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b69bd56-7a7e-4136-ad29-210d13fdbaa1 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:14.554043 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-0b69bd56-7a7e-4136-ad29-210d13fdbaa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:18:14.570482 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b69bd56-7a7e-4136-ad29-210d13fdbaa1 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:14.576240 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb6f05-b5a2-4755-b523-b93a638f16dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:14.577009 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb6f05-b5a2-4755-b523-b93a638f16dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:14.577123 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb6f05-b5a2-4755-b523-b93a638f16dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:14.577461 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb6f05-b5a2-4755-b523-b93a638f16dd admin admin] PATH_INFO: `/v3/projects/896c1d3b2d664d19bf6afc0f8bd19ed7` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:14.578608 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb6f05-b5a2-4755-b523-b93a638f16dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=896c1d3b2d664d19bf6afc0f8bd19ed7)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:14.580458 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb6f05-b5a2-4755-b523-b93a638f16dd admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:14.627125 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-e1fb6f05-b5a2-4755-b523-b93a638f16dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '896c1d3b2d664d19bf6afc0f8bd19ed7', 'request_id': 'req-e1fb6f05-b5a2-4755-b523-b93a638f16dd'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:14.632612 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-e1fb6f05-b5a2-4755-b523-b93a638f16dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '896c1d3b2d664d19bf6afc0f8bd19ed7', 'request_id': 'req-e1fb6f05-b5a2-4755-b523-b93a638f16dd'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:14.639310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-e1fb6f05-b5a2-4755-b523-b93a638f16dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '896c1d3b2d664d19bf6afc0f8bd19ed7', 'request_id': 'req-e1fb6f05-b5a2-4755-b523-b93a638f16dd'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:14.651579 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-e1fb6f05-b5a2-4755-b523-b93a638f16dd admin admin] The token cache is being invalidate because project 896c1d3b2d664d19bf6afc0f8bd19ed7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:18:14.651887 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-e1fb6f05-b5a2-4755-b523-b93a638f16dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:14.652695 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-e1fb6f05-b5a2-4755-b523-b93a638f16dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:14.654189 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1404/2791] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 15:18:14 2022] DELETE /identity/v3/projects/896c1d3b2d664d19bf6afc0f8bd19ed7 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:18:14.658998 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99bc292-dd36-48b9-942e-b6e10566ab80 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:14.659510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99bc292-dd36-48b9-942e-b6e10566ab80 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:14.659613 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99bc292-dd36-48b9-942e-b6e10566ab80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:14.678765 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-a99bc292-dd36-48b9-942e-b6e10566ab80 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:18:14.679999 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-a99bc292-dd36-48b9-942e-b6e10566ab80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:18:14.739116 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1388/2792] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:18:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:18:14.747362 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0edb43f-acb6-4306-bc8d-0bdbf96f0e06 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:14.747652 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0edb43f-acb6-4306-bc8d-0bdbf96f0e06 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:14.747925 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0edb43f-acb6-4306-bc8d-0bdbf96f0e06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:14.767901 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-d0edb43f-acb6-4306-bc8d-0bdbf96f0e06 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:18:14.769101 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-d0edb43f-acb6-4306-bc8d-0bdbf96f0e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:18:14.776994 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1405/2793] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:18:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:18:14.781143 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99bc292-dd36-48b9-942e-b6e10566ab80 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:14.790972 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99bc292-dd36-48b9-942e-b6e10566ab80 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:14.796170 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aeb16b-63d7-41aa-bacc-f51ed058d246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:14.796765 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aeb16b-63d7-41aa-bacc-f51ed058d246 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:14.797047 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aeb16b-63d7-41aa-bacc-f51ed058d246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:14.797319 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aeb16b-63d7-41aa-bacc-f51ed058d246 admin admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:14.797968 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aeb16b-63d7-41aa-bacc-f51ed058d246 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:18:14.798270 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aeb16b-63d7-41aa-bacc-f51ed058d246 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:14.799916 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aeb16b-63d7-41aa-bacc-f51ed058d246 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:14.811982 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1389/2794] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 15:18:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:18:14.816173 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0edb43f-acb6-4306-bc8d-0bdbf96f0e06 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:14.825886 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0edb43f-acb6-4306-bc8d-0bdbf96f0e06 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:14.830730 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91cdfb5-07a7-47c0-8c20-18f8a2817941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:14.831376 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91cdfb5-07a7-47c0-8c20-18f8a2817941 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:14.831659 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91cdfb5-07a7-47c0-8c20-18f8a2817941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:14.831934 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91cdfb5-07a7-47c0-8c20-18f8a2817941 admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:14.832566 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91cdfb5-07a7-47c0-8c20-18f8a2817941 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:14.834458 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91cdfb5-07a7-47c0-8c20-18f8a2817941 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:14.843978 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:18:14.862583 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1406/2795] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 15:18:14 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:18:14.866582 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aeb16b-63d7-41aa-bacc-f51ed058d246 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:14.877185 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aeb16b-63d7-41aa-bacc-f51ed058d246 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:14.932512 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6f889a-11fe-4870-b81c-604debee4cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:14.933115 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6f889a-11fe-4870-b81c-604debee4cbd admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:14.933960 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6f889a-11fe-4870-b81c-604debee4cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:14.934254 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6f889a-11fe-4870-b81c-604debee4cbd admin admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:14.935002 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6f889a-11fe-4870-b81c-604debee4cbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:14.936956 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6f889a-11fe-4870-b81c-604debee4cbd admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:14.957915 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1390/2796] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 15:18:14 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:18:14.962029 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91cdfb5-07a7-47c0-8c20-18f8a2817941 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:14.972312 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91cdfb5-07a7-47c0-8c20-18f8a2817941 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:14.976931 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2d5c18-400e-4ec5-801e-9a9bb786c4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:14.977521 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d5c18-400e-4ec5-801e-9a9bb786c4d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:14.977796 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d5c18-400e-4ec5-801e-9a9bb786c4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:14.978063 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d5c18-400e-4ec5-801e-9a9bb786c4d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:14.978695 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2d5c18-400e-4ec5-801e-9a9bb786c4d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:14.980958 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2d5c18-400e-4ec5-801e-9a9bb786c4d0 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:14.990803 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1407/2797] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:18:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:18:14.994609 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6f889a-11fe-4870-b81c-604debee4cbd admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:15.005218 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6f889a-11fe-4870-b81c-604debee4cbd admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:15.010213 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e9e05-375c-4cc0-a3ba-1d7a7756acd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:15.010814 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e9e05-375c-4cc0-a3ba-1d7a7756acd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:15.011096 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e9e05-375c-4cc0-a3ba-1d7a7756acd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:15.011500 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e9e05-375c-4cc0-a3ba-1d7a7756acd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:15.012334 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178e9e05-375c-4cc0-a3ba-1d7a7756acd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:15.013926 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178e9e05-375c-4cc0-a3ba-1d7a7756acd7 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:15.025688 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1391/2798] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:18:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:18:15.030161 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2d5c18-400e-4ec5-801e-9a9bb786c4d0 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:15.040171 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2d5c18-400e-4ec5-801e-9a9bb786c4d0 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:15.045387 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e93a2b-82f3-403c-9582-4c7ed59924a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:15.046020 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e93a2b-82f3-403c-9582-4c7ed59924a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:15.046295 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e93a2b-82f3-403c-9582-4c7ed59924a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:15.046640 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e93a2b-82f3-403c-9582-4c7ed59924a4 admin admin] PATH_INFO: `/v3/projects/4edd140cb317483e8e7e3f04cdee048c/users/93a00d9af55a42fa8d5628fa81dcae80/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:15.061954 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e93a2b-82f3-403c-9582-4c7ed59924a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4edd140cb317483e8e7e3f04cdee048c, user_id=93a00d9af55a42fa8d5628fa81dcae80, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:15.064625 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e93a2b-82f3-403c-9582-4c7ed59924a4 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:15.078114 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1408/2799] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 15:18:15 2022] PUT /identity/v3/projects/4edd140cb317483e8e7e3f04cdee048c/users/93a00d9af55a42fa8d5628fa81dcae80/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:18:15.083426 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df30340f-41dc-4d6e-8597-273dcdd1fe2e None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:15.083719 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df30340f-41dc-4d6e-8597-273dcdd1fe2e None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:15.083997 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df30340f-41dc-4d6e-8597-273dcdd1fe2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:15.103110 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-df30340f-41dc-4d6e-8597-273dcdd1fe2e None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:18:15.104348 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-df30340f-41dc-4d6e-8597-273dcdd1fe2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:18:15.114336 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:18:15.164328 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1392/2800] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:18:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:18:15.174814 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e93a2b-82f3-403c-9582-4c7ed59924a4 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:15.176678 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-80e93a2b-82f3-403c-9582-4c7ed59924a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:18:15.234802 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e93a2b-82f3-403c-9582-4c7ed59924a4 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:15.238474 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e477d7-c37f-4ed8-95b4-e57608d289b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:15.239077 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e477d7-c37f-4ed8-95b4-e57608d289b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:15.239387 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e477d7-c37f-4ed8-95b4-e57608d289b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:15.239659 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e477d7-c37f-4ed8-95b4-e57608d289b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:15.249154 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e477d7-c37f-4ed8-95b4-e57608d289b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:15.250503 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e477d7-c37f-4ed8-95b4-e57608d289b6 service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:15.259796 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1409/2801] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:18:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:18:19.291688 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a6655-d5e0-44bd-933d-1b553b488845 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:19.292157 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a6655-d5e0-44bd-933d-1b553b488845 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:19.292264 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a6655-d5e0-44bd-933d-1b553b488845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:19.334149 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-995a6655-d5e0-44bd-933d-1b553b488845 None None] MFA Rules not processed for user `93a00d9af55a42fa8d5628fa81dcae80`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:18:19.335520 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-995a6655-d5e0-44bd-933d-1b553b488845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:18:19.439532 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1393/2802] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:18:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:18:19.440524 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e477d7-c37f-4ed8-95b4-e57608d289b6 service neutron] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:19.452635 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e477d7-c37f-4ed8-95b4-e57608d289b6 service neutron] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:19.457406 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df553c58-2035-47f6-8616-682a444769da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:19.457948 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df553c58-2035-47f6-8616-682a444769da admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:19.458229 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df553c58-2035-47f6-8616-682a444769da admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:19.458498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df553c58-2035-47f6-8616-682a444769da admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:19.459124 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df553c58-2035-47f6-8616-682a444769da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:19.461071 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df553c58-2035-47f6-8616-682a444769da admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:19.491283 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1410/2803] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 15:18:19 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:18:19.498864 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a6655-d5e0-44bd-933d-1b553b488845 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:19.529235 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a6655-d5e0-44bd-933d-1b553b488845 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:19.604462 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b3c1a8-7294-41c4-ba21-59542216488d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:19.605269 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b3c1a8-7294-41c4-ba21-59542216488d admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:19.605538 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b3c1a8-7294-41c4-ba21-59542216488d admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:19.605908 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b3c1a8-7294-41c4-ba21-59542216488d admin admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:19.606626 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b3c1a8-7294-41c4-ba21-59542216488d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:19.608812 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b3c1a8-7294-41c4-ba21-59542216488d admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:19.633969 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1394/2804] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 15:18:19 2022] POST /identity/v3/users => generated 299 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:18:19.638951 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df553c58-2035-47f6-8616-682a444769da admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:19.650491 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df553c58-2035-47f6-8616-682a444769da admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:19.655504 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce07a210-a9a8-4cec-bc88-f9b22bd3f3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:19.656115 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07a210-a9a8-4cec-bc88-f9b22bd3f3de admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:19.656395 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07a210-a9a8-4cec-bc88-f9b22bd3f3de admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:19.656667 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07a210-a9a8-4cec-bc88-f9b22bd3f3de admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:19.657503 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce07a210-a9a8-4cec-bc88-f9b22bd3f3de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:19.659301 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce07a210-a9a8-4cec-bc88-f9b22bd3f3de admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:19.679755 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1411/2805] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:18:19 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 15:18:19.685139 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b3c1a8-7294-41c4-ba21-59542216488d admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:19.695680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b3c1a8-7294-41c4-ba21-59542216488d admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:19.701804 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a853c0-a44f-40de-a18a-3ee8cc672c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:19.702484 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a853c0-a44f-40de-a18a-3ee8cc672c86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:19.702780 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a853c0-a44f-40de-a18a-3ee8cc672c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:19.703189 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a853c0-a44f-40de-a18a-3ee8cc672c86 admin admin] PATH_INFO: `/v3/projects/b2a376925fda431ea5ce67cc7896d1a6/users/519d88ca754b4a23a9b7958a87c40150/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:19.720919 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a853c0-a44f-40de-a18a-3ee8cc672c86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2a376925fda431ea5ce67cc7896d1a6, user_id=519d88ca754b4a23a9b7958a87c40150, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:19.723743 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a853c0-a44f-40de-a18a-3ee8cc672c86 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:19.738712 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1395/2806] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 15:18:19 2022] PUT /identity/v3/projects/b2a376925fda431ea5ce67cc7896d1a6/users/519d88ca754b4a23a9b7958a87c40150/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:18:19.742718 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce07a210-a9a8-4cec-bc88-f9b22bd3f3de admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:19.756443 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce07a210-a9a8-4cec-bc88-f9b22bd3f3de admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:19.827144 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be345b5-dba6-4da4-a062-af0cae5152dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:19.827144 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be345b5-dba6-4da4-a062-af0cae5152dc admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:19.827557 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be345b5-dba6-4da4-a062-af0cae5152dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:19.827693 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be345b5-dba6-4da4-a062-af0cae5152dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:19.828350 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be345b5-dba6-4da4-a062-af0cae5152dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:19.829968 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be345b5-dba6-4da4-a062-af0cae5152dc admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:19.851417 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1412/2807] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:18:19 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:18:19.852342 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a853c0-a44f-40de-a18a-3ee8cc672c86 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:19.863925 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a853c0-a44f-40de-a18a-3ee8cc672c86 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:19.868978 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7cb564-5111-4793-b698-679228eb8d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:19.869746 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7cb564-5111-4793-b698-679228eb8d43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:19.870108 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7cb564-5111-4793-b698-679228eb8d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:19.870445 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7cb564-5111-4793-b698-679228eb8d43 admin admin] PATH_INFO: `/v3/projects/b2a376925fda431ea5ce67cc7896d1a6/users/519d88ca754b4a23a9b7958a87c40150/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:19.873718 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7cb564-5111-4793-b698-679228eb8d43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2a376925fda431ea5ce67cc7896d1a6, user_id=519d88ca754b4a23a9b7958a87c40150, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:19.876437 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7cb564-5111-4793-b698-679228eb8d43 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:19.882961 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:18:19.895148 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1396/2808] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 15:18:19 2022] PUT /identity/v3/projects/b2a376925fda431ea5ce67cc7896d1a6/users/519d88ca754b4a23a9b7958a87c40150/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:18:25.174406 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a34d5-5af2-41bf-8fea-5fb953b2b610 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:25.174948 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a34d5-5af2-41bf-8fea-5fb953b2b610 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:25.175113 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a34d5-5af2-41bf-8fea-5fb953b2b610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:25.204364 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-ff3a34d5-5af2-41bf-8fea-5fb953b2b610 None None] MFA Rules not processed for user `519d88ca754b4a23a9b7958a87c40150`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:18:25.205827 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-ff3a34d5-5af2-41bf-8fea-5fb953b2b610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:18:25.216537 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:18:25.319263 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1413/2809] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:18:25 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:18:25.333514 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7cb564-5111-4793-b698-679228eb8d43 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:25.335397 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-cb7cb564-5111-4793-b698-679228eb8d43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:18:25.429061 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7cb564-5111-4793-b698-679228eb8d43 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:25.435988 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a0755a-aa79-4a8b-8c73-887f52335543 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6889402e54424bbd8d42c7bb759cf700', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:25.436557 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0755a-aa79-4a8b-8c73-887f52335543 service glance] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:25.436837 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0755a-aa79-4a8b-8c73-887f52335543 service glance] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:25.437486 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0755a-aa79-4a8b-8c73-887f52335543 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:25.514947 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a0755a-aa79-4a8b-8c73-887f52335543 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:25.516493 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a0755a-aa79-4a8b-8c73-887f52335543 service glance] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:25.526408 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1397/2810] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:18:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:18:25.716626 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d770d2-e7ff-40a6-be40-de6f9a8ca112 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:25.716915 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d770d2-e7ff-40a6-be40-de6f9a8ca112 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:25.717217 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d770d2-e7ff-40a6-be40-de6f9a8ca112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:25.814671 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-12d770d2-e7ff-40a6-be40-de6f9a8ca112 None None] MFA Rules not processed for user `f60bee471e954932a928ab2cf7d66c82`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:18:25.818053 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-12d770d2-e7ff-40a6-be40-de6f9a8ca112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:18:25.905969 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1414/2811] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Aug 30 15:18:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:18:25.958770 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a0755a-aa79-4a8b-8c73-887f52335543 service glance] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:25.960344 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-b7a0755a-aa79-4a8b-8c73-887f52335543 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:18:26.086792 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a0755a-aa79-4a8b-8c73-887f52335543 service glance] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:26.093118 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf0ec78-f126-4835-a668-b9d45eae931e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a18b13e3708144e1b5e8df80ec3a4e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:26.093967 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0ec78-f126-4835-a668-b9d45eae931e service swift] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:26.094410 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0ec78-f126-4835-a668-b9d45eae931e service swift] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:26.094823 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0ec78-f126-4835-a668-b9d45eae931e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:26.104014 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf0ec78-f126-4835-a668-b9d45eae931e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:26.105881 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf0ec78-f126-4835-a668-b9d45eae931e service swift] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:26.122017 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1398/2812] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Aug 30 15:18:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 15:18:35.024368 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60561c0-6224-4eb4-9e2f-8bae83602ea5 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:35.024917 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60561c0-6224-4eb4-9e2f-8bae83602ea5 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:35.024917 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60561c0-6224-4eb4-9e2f-8bae83602ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:35.044928 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-f60561c0-6224-4eb4-9e2f-8bae83602ea5 None None] MFA Rules not processed for user `93a00d9af55a42fa8d5628fa81dcae80`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:18:35.046130 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-f60561c0-6224-4eb4-9e2f-8bae83602ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:18:35.053617 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1415/2813] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:18:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:18:35.323945 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf0ec78-f126-4835-a668-b9d45eae931e service swift] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:35.325707 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-ccf0ec78-f126-4835-a668-b9d45eae931e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:18:35.331043 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:18:35.383570 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf0ec78-f126-4835-a668-b9d45eae931e service swift] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:35.387342 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ec68b1-bd5f-472c-b3e6-a6519b3915ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:35.387946 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec68b1-bd5f-472c-b3e6-a6519b3915ce service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:35.388228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec68b1-bd5f-472c-b3e6-a6519b3915ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:35.388504 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec68b1-bd5f-472c-b3e6-a6519b3915ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:35.390205 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-21ec68b1-bd5f-472c-b3e6-a6519b3915ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:18:35.399577 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ec68b1-bd5f-472c-b3e6-a6519b3915ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:35.401008 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ec68b1-bd5f-472c-b3e6-a6519b3915ce service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:35.409403 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1399/2814] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:18:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:18:36.086930 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60561c0-6224-4eb4-9e2f-8bae83602ea5 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:36.098504 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60561c0-6224-4eb4-9e2f-8bae83602ea5 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:36.104962 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca13e9-1d90-49a8-9987-da77766e14e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:36.106384 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca13e9-1d90-49a8-9987-da77766e14e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:36.106672 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca13e9-1d90-49a8-9987-da77766e14e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:36.106976 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca13e9-1d90-49a8-9987-da77766e14e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:36.108794 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-01ca13e9-1d90-49a8-9987-da77766e14e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:18:36.176786 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=100 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:18:36.179741 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ca13e9-1d90-49a8-9987-da77766e14e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:36.181493 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ca13e9-1d90-49a8-9987-da77766e14e0 service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:36.193944 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1416/2815] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:18:36 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 15:18:37.213377 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac5189-ca54-48ec-8045-bfa5d4e95fb0 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:37.214416 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac5189-ca54-48ec-8045-bfa5d4e95fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:37.214708 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac5189-ca54-48ec-8045-bfa5d4e95fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:37.235866 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-d0ac5189-ca54-48ec-8045-bfa5d4e95fb0 None None] MFA Rules not processed for user `f60bee471e954932a928ab2cf7d66c82`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:18:37.237290 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-d0ac5189-ca54-48ec-8045-bfa5d4e95fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:18:37.244610 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1400/2816] 10.222.0.30 () {58 vars in 1009 bytes} [Tue Aug 30 15:18:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:18:37.260379 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca13e9-1d90-49a8-9987-da77766e14e0 service neutron] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:37.272834 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca13e9-1d90-49a8-9987-da77766e14e0 service neutron] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:37.279111 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1307e7f1-69a8-44dc-ab5b-f8c1d68a10ce service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a18b13e3708144e1b5e8df80ec3a4e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:37.279760 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307e7f1-69a8-44dc-ab5b-f8c1d68a10ce service swift] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:37.280040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307e7f1-69a8-44dc-ab5b-f8c1d68a10ce service swift] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:37.280312 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307e7f1-69a8-44dc-ab5b-f8c1d68a10ce service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:37.288730 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1307e7f1-69a8-44dc-ab5b-f8c1d68a10ce service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:37.290300 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1307e7f1-69a8-44dc-ab5b-f8c1d68a10ce service swift] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:37.298911 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1417/2817] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Aug 30 15:18:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 15:18:38.288626 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac5189-ca54-48ec-8045-bfa5d4e95fb0 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:38.290322 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-d0ac5189-ca54-48ec-8045-bfa5d4e95fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:18:38.391619 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac5189-ca54-48ec-8045-bfa5d4e95fb0 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:38.397393 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f28a6e-1bb3-4ab6-91c8-d5d9a5d0c96a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:38.398030 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f28a6e-1bb3-4ab6-91c8-d5d9a5d0c96a service nova] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:38.398316 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f28a6e-1bb3-4ab6-91c8-d5d9a5d0c96a service nova] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:38.398591 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f28a6e-1bb3-4ab6-91c8-d5d9a5d0c96a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:38.400421 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-04f28a6e-1bb3-4ab6-91c8-d5d9a5d0c96a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:18:38.414304 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f28a6e-1bb3-4ab6-91c8-d5d9a5d0c96a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:38.415964 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f28a6e-1bb3-4ab6-91c8-d5d9a5d0c96a service nova] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:38.430209 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1401/2818] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:18:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:18:47.295513 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=8 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:18:48.425046 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=62 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:18:53.405427 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1307e7f1-69a8-44dc-ab5b-f8c1d68a10ce service swift] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:53.414772 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1307e7f1-69a8-44dc-ab5b-f8c1d68a10ce service swift] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:53.418910 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f178c5-5439-4ec3-8881-03218ec98091 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:53.419533 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f178c5-5439-4ec3-8881-03218ec98091 service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:53.419811 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f178c5-5439-4ec3-8881-03218ec98091 service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:53.420079 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f178c5-5439-4ec3-8881-03218ec98091 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:53.421820 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-28f178c5-5439-4ec3-8881-03218ec98091 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:18:53.485204 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f178c5-5439-4ec3-8881-03218ec98091 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:53.486697 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f178c5-5439-4ec3-8881-03218ec98091 service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:53.497123 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1418/2819] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:18:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:18:57.418161 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f28a6e-1bb3-4ab6-91c8-d5d9a5d0c96a service nova] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:57.419888 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-04f28a6e-1bb3-4ab6-91c8-d5d9a5d0c96a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:18:57.434793 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f28a6e-1bb3-4ab6-91c8-d5d9a5d0c96a service nova] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:57.439743 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61fde7e-c027-416f-a8ee-2f067220053e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:57.440407 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61fde7e-c027-416f-a8ee-2f067220053e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:57.440689 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61fde7e-c027-416f-a8ee-2f067220053e admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:57.441089 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61fde7e-c027-416f-a8ee-2f067220053e admin admin] PATH_INFO: `/v3/users/ecec9a5a18b24ae7ab055c37ea85ca18` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:57.442232 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61fde7e-c027-416f-a8ee-2f067220053e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecec9a5a18b24ae7ab055c37ea85ca18)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:57.444195 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61fde7e-c027-416f-a8ee-2f067220053e admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:57.494535 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b61fde7e-c027-416f-a8ee-2f067220053e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecec9a5a18b24ae7ab055c37ea85ca18'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:57.495173 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b61fde7e-c027-416f-a8ee-2f067220053e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecec9a5a18b24ae7ab055c37ea85ca18'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:57.507093 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b61fde7e-c027-416f-a8ee-2f067220053e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecec9a5a18b24ae7ab055c37ea85ca18'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:57.514901 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b61fde7e-c027-416f-a8ee-2f067220053e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecec9a5a18b24ae7ab055c37ea85ca18'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:57.522226 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b61fde7e-c027-416f-a8ee-2f067220053e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecec9a5a18b24ae7ab055c37ea85ca18'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:57.529189 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1402/2820] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 15:18:57 2022] DELETE /identity/v3/users/ecec9a5a18b24ae7ab055c37ea85ca18 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:18:57.700429 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f178c5-5439-4ec3-8881-03218ec98091 service neutron] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:57.710673 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f178c5-5439-4ec3-8881-03218ec98091 service neutron] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:57.769446 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a18c2b4-c59d-4a7e-8704-6ccfa6bc3cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:57.770078 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18c2b4-c59d-4a7e-8704-6ccfa6bc3cec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:57.770352 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18c2b4-c59d-4a7e-8704-6ccfa6bc3cec admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:57.770629 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18c2b4-c59d-4a7e-8704-6ccfa6bc3cec admin admin] PATH_INFO: `/v3/projects/1827feb276a945bb86a2f853126dfb20` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:57.771776 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a18c2b4-c59d-4a7e-8704-6ccfa6bc3cec admin admin] RBAC: Authorizing `identity:delete_project(project_id=1827feb276a945bb86a2f853126dfb20)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:57.774031 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a18c2b4-c59d-4a7e-8704-6ccfa6bc3cec admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:57.821320 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-8a18c2b4-c59d-4a7e-8704-6ccfa6bc3cec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1827feb276a945bb86a2f853126dfb20', 'request_id': 'req-8a18c2b4-c59d-4a7e-8704-6ccfa6bc3cec'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:57.827310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-8a18c2b4-c59d-4a7e-8704-6ccfa6bc3cec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1827feb276a945bb86a2f853126dfb20', 'request_id': 'req-8a18c2b4-c59d-4a7e-8704-6ccfa6bc3cec'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:57.833389 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-8a18c2b4-c59d-4a7e-8704-6ccfa6bc3cec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1827feb276a945bb86a2f853126dfb20', 'request_id': 'req-8a18c2b4-c59d-4a7e-8704-6ccfa6bc3cec'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:57.846149 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-8a18c2b4-c59d-4a7e-8704-6ccfa6bc3cec admin admin] The token cache is being invalidate because project 1827feb276a945bb86a2f853126dfb20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:18:57.846453 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-8a18c2b4-c59d-4a7e-8704-6ccfa6bc3cec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:57.847117 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-8a18c2b4-c59d-4a7e-8704-6ccfa6bc3cec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:18:57.848977 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1419/2821] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 15:18:57 2022] DELETE /identity/v3/projects/1827feb276a945bb86a2f853126dfb20 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:18:57.854208 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c15e62-c38b-4bc5-8d4e-70c1d89406c8 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:57.854607 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c15e62-c38b-4bc5-8d4e-70c1d89406c8 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:57.854951 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c15e62-c38b-4bc5-8d4e-70c1d89406c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:57.875066 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-a6c15e62-c38b-4bc5-8d4e-70c1d89406c8 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:18:57.876330 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-a6c15e62-c38b-4bc5-8d4e-70c1d89406c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:18:57.937355 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1403/2822] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:18:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 15:18:57.945103 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f6fd6-d018-47b6-a946-3190072c35ee None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:57.945388 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f6fd6-d018-47b6-a946-3190072c35ee None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:57.945667 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f6fd6-d018-47b6-a946-3190072c35ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:57.966602 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-245f6fd6-d018-47b6-a946-3190072c35ee None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:18:57.967844 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-245f6fd6-d018-47b6-a946-3190072c35ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:18:57.976093 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1420/2823] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:18:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:18:57.980487 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c15e62-c38b-4bc5-8d4e-70c1d89406c8 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:57.991000 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c15e62-c38b-4bc5-8d4e-70c1d89406c8 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:57.995860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e05dd0-3e02-4053-85da-9bdfd83b14b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:57.996539 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e05dd0-3e02-4053-85da-9bdfd83b14b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:57.996910 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e05dd0-3e02-4053-85da-9bdfd83b14b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:57.997255 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e05dd0-3e02-4053-85da-9bdfd83b14b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:57.997978 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e05dd0-3e02-4053-85da-9bdfd83b14b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:18:57.998353 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e05dd0-3e02-4053-85da-9bdfd83b14b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:58.000149 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e05dd0-3e02-4053-85da-9bdfd83b14b9 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:58.016262 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1404/2824] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 15:18:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:18:58.017707 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245f6fd6-d018-47b6-a946-3190072c35ee None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:58.027891 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245f6fd6-d018-47b6-a946-3190072c35ee None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:58.032640 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0489bf-76df-4d00-91d4-9978856e4291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:58.033296 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0489bf-76df-4d00-91d4-9978856e4291 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:58.033572 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0489bf-76df-4d00-91d4-9978856e4291 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:58.033840 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0489bf-76df-4d00-91d4-9978856e4291 admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:58.034467 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0489bf-76df-4d00-91d4-9978856e4291 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:58.036399 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0489bf-76df-4d00-91d4-9978856e4291 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:58.052546 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:18:58.066544 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1421/2825] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 15:18:58 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:18:58.070868 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e05dd0-3e02-4053-85da-9bdfd83b14b9 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:58.082376 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e05dd0-3e02-4053-85da-9bdfd83b14b9 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:58.140361 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb9db60-8c82-4ea2-be19-344a43efe3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:58.140967 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb9db60-8c82-4ea2-be19-344a43efe3f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:58.141244 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb9db60-8c82-4ea2-be19-344a43efe3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:58.141516 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb9db60-8c82-4ea2-be19-344a43efe3f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:58.142142 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb9db60-8c82-4ea2-be19-344a43efe3f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:58.144092 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb9db60-8c82-4ea2-be19-344a43efe3f2 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:58.153406 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:18:58.169146 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1405/2826] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 15:18:58 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:18:58.173429 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0489bf-76df-4d00-91d4-9978856e4291 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:58.183809 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0489bf-76df-4d00-91d4-9978856e4291 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:58.188655 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ebef1-cad8-47b5-b847-ccf86338d75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:58.189280 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ebef1-cad8-47b5-b847-ccf86338d75d admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:58.189558 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ebef1-cad8-47b5-b847-ccf86338d75d admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:58.189829 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ebef1-cad8-47b5-b847-ccf86338d75d admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:58.190470 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0ebef1-cad8-47b5-b847-ccf86338d75d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:58.192119 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0ebef1-cad8-47b5-b847-ccf86338d75d admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:58.203753 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1422/2827] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:18:58.207842 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb9db60-8c82-4ea2-be19-344a43efe3f2 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:58.219063 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb9db60-8c82-4ea2-be19-344a43efe3f2 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:58.224700 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628fc121-6c7b-403d-8a3c-e1040924179d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:58.225307 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628fc121-6c7b-403d-8a3c-e1040924179d admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:58.225588 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628fc121-6c7b-403d-8a3c-e1040924179d admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:58.225862 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628fc121-6c7b-403d-8a3c-e1040924179d admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:58.226503 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628fc121-6c7b-403d-8a3c-e1040924179d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:58.228135 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628fc121-6c7b-403d-8a3c-e1040924179d admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:58.239597 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1406/2828] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:18:58.243746 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ebef1-cad8-47b5-b847-ccf86338d75d admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:58.254008 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ebef1-cad8-47b5-b847-ccf86338d75d admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:58.258561 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653612b2-ff88-43fc-8d6e-79c71df39c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:58.259201 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653612b2-ff88-43fc-8d6e-79c71df39c21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:58.259505 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653612b2-ff88-43fc-8d6e-79c71df39c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:58.260258 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653612b2-ff88-43fc-8d6e-79c71df39c21 admin admin] PATH_INFO: `/v3/projects/49ebb168eca54a0fac401af8f3e4e46e/users/f6d2b34ebaaf4e578dea262d52989e07/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:58.274528 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653612b2-ff88-43fc-8d6e-79c71df39c21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49ebb168eca54a0fac401af8f3e4e46e, user_id=f6d2b34ebaaf4e578dea262d52989e07, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:58.277744 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653612b2-ff88-43fc-8d6e-79c71df39c21 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:58.290454 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1423/2829] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 15:18:58 2022] PUT /identity/v3/projects/49ebb168eca54a0fac401af8f3e4e46e/users/f6d2b34ebaaf4e578dea262d52989e07/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:18:58.300620 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe93572-df7a-4c89-9baf-f9315c7d3b82 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:58.300919 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe93572-df7a-4c89-9baf-f9315c7d3b82 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:58.301174 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe93572-df7a-4c89-9baf-f9315c7d3b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:58.331734 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-bfe93572-df7a-4c89-9baf-f9315c7d3b82 None None] MFA Rules not processed for user `f6d2b34ebaaf4e578dea262d52989e07`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:18:58.333027 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-bfe93572-df7a-4c89-9baf-f9315c7d3b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:18:58.397855 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1407/2830] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:18:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:18:58.402494 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653612b2-ff88-43fc-8d6e-79c71df39c21 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:58.467137 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653612b2-ff88-43fc-8d6e-79c71df39c21 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:58.471802 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bb024e-e0e4-49e7-bb4e-20e700f65249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:58.472407 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bb024e-e0e4-49e7-bb4e-20e700f65249 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:58.472687 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bb024e-e0e4-49e7-bb4e-20e700f65249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:58.472959 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bb024e-e0e4-49e7-bb4e-20e700f65249 admin admin] PATH_INFO: `/v3/projects` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:58.473627 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bb024e-e0e4-49e7-bb4e-20e700f65249 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:58.475578 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bb024e-e0e4-49e7-bb4e-20e700f65249 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:58.504373 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1424/2831] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 15:18:58 2022] POST /identity/v3/projects => generated 375 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:18:58.508400 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe93572-df7a-4c89-9baf-f9315c7d3b82 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:58.520072 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe93572-df7a-4c89-9baf-f9315c7d3b82 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:58.581501 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34096e49-635a-499a-b3ca-8d074cb9bdaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:58.582028 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34096e49-635a-499a-b3ca-8d074cb9bdaf admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:58.582307 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34096e49-635a-499a-b3ca-8d074cb9bdaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:58.582580 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34096e49-635a-499a-b3ca-8d074cb9bdaf admin admin] PATH_INFO: `/v3/users` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:58.583527 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34096e49-635a-499a-b3ca-8d074cb9bdaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:58.585566 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34096e49-635a-499a-b3ca-8d074cb9bdaf admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:58.608340 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1408/2832] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 15:18:58 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:18:58.612448 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bb024e-e0e4-49e7-bb4e-20e700f65249 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:58.623928 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bb024e-e0e4-49e7-bb4e-20e700f65249 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:58.628678 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42985036-0204-4933-9471-e8231e9a8a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:58.629435 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42985036-0204-4933-9471-e8231e9a8a63 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:58.629713 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42985036-0204-4933-9471-e8231e9a8a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:58.629983 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42985036-0204-4933-9471-e8231e9a8a63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:58.630621 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42985036-0204-4933-9471-e8231e9a8a63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:58.632250 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42985036-0204-4933-9471-e8231e9a8a63 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:58.643927 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1425/2833] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:18:58.648380 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34096e49-635a-499a-b3ca-8d074cb9bdaf admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:58.658668 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34096e49-635a-499a-b3ca-8d074cb9bdaf admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:58.663770 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629f35c5-639c-4121-97d4-f05a4d2f4424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:58.664433 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f35c5-639c-4121-97d4-f05a4d2f4424 admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:58.664715 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f35c5-639c-4121-97d4-f05a4d2f4424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:58.664989 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f35c5-639c-4121-97d4-f05a4d2f4424 admin admin] PATH_INFO: `/v3/projects/24879e909603453a84d215121b0f73cb/users/3ce4293863dd4ec0b3d4198ea7aa639d/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:58.680599 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629f35c5-639c-4121-97d4-f05a4d2f4424 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24879e909603453a84d215121b0f73cb, user_id=3ce4293863dd4ec0b3d4198ea7aa639d, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:58.684066 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629f35c5-639c-4121-97d4-f05a4d2f4424 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:58.700396 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1409/2834] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 15:18:58 2022] PUT /identity/v3/projects/24879e909603453a84d215121b0f73cb/users/3ce4293863dd4ec0b3d4198ea7aa639d/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:18:58.701573 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42985036-0204-4933-9471-e8231e9a8a63 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:58.712256 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42985036-0204-4933-9471-e8231e9a8a63 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:58.772560 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149f8578-4bee-4483-9eb8-1b25b9522f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:58.773170 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f8578-4bee-4483-9eb8-1b25b9522f89 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:58.773471 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f8578-4bee-4483-9eb8-1b25b9522f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:58.773727 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f8578-4bee-4483-9eb8-1b25b9522f89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:58.774362 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149f8578-4bee-4483-9eb8-1b25b9522f89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:58.776130 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149f8578-4bee-4483-9eb8-1b25b9522f89 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:58.786212 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1426/2835] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:18:58.790471 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629f35c5-639c-4121-97d4-f05a4d2f4424 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:58.801406 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629f35c5-639c-4121-97d4-f05a4d2f4424 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:58.806086 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cacb71-68ea-4450-b4dd-d3f3f1867b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:58.806880 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cacb71-68ea-4450-b4dd-d3f3f1867b9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:58.807245 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cacb71-68ea-4450-b4dd-d3f3f1867b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:58.807580 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cacb71-68ea-4450-b4dd-d3f3f1867b9d admin admin] PATH_INFO: `/v3/projects/24879e909603453a84d215121b0f73cb/users/3ce4293863dd4ec0b3d4198ea7aa639d/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:58.809547 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cacb71-68ea-4450-b4dd-d3f3f1867b9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=24879e909603453a84d215121b0f73cb, user_id=3ce4293863dd4ec0b3d4198ea7aa639d, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:58.812271 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cacb71-68ea-4450-b4dd-d3f3f1867b9d admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:58.823099 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1410/2836] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 15:18:58 2022] PUT /identity/v3/projects/24879e909603453a84d215121b0f73cb/users/3ce4293863dd4ec0b3d4198ea7aa639d/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:18:58.890394 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d791b-c02b-4937-9b21-58265163295b None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:58.890671 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d791b-c02b-4937-9b21-58265163295b None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:58.890950 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d791b-c02b-4937-9b21-58265163295b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:58.919372 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-c94d791b-c02b-4937-9b21-58265163295b None None] MFA Rules not processed for user `3ce4293863dd4ec0b3d4198ea7aa639d`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:18:58.920610 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-c94d791b-c02b-4937-9b21-58265163295b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:18:58.940325 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:18:58.991354 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1427/2837] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:18:58 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:18:59.002145 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cacb71-68ea-4450-b4dd-d3f3f1867b9d admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:18:59.003833 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-29cacb71-68ea-4450-b4dd-d3f3f1867b9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:18:59.027904 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:18:59.067059 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cacb71-68ea-4450-b4dd-d3f3f1867b9d admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:18:59.070980 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bcd319-9564-451b-ab9c-f96c01fe6bd2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:18:59.071636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bcd319-9564-451b-ab9c-f96c01fe6bd2 service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:18:59.071916 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bcd319-9564-451b-ab9c-f96c01fe6bd2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:18:59.072190 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bcd319-9564-451b-ab9c-f96c01fe6bd2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:18:59.133740 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bcd319-9564-451b-ab9c-f96c01fe6bd2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:18:59.136424 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bcd319-9564-451b-ab9c-f96c01fe6bd2 service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:18:59.145939 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1411/2838] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:18:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:19:04.814160 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94d791b-c02b-4937-9b21-58265163295b None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:19:04.824171 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94d791b-c02b-4937-9b21-58265163295b None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:19:04.828058 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5dfc7-8634-479c-89cd-0e40f649b527 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:19:04.828654 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5dfc7-8634-479c-89cd-0e40f649b527 service cinder] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:04.828936 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5dfc7-8634-479c-89cd-0e40f649b527 service cinder] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:04.829218 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5dfc7-8634-479c-89cd-0e40f649b527 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:04.838238 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c5dfc7-8634-479c-89cd-0e40f649b527 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:19:04.839793 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c5dfc7-8634-479c-89cd-0e40f649b527 service cinder] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:19:04.849417 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1428/2839] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:19:04 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:19:09.141471 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=39 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:19:11.881954 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bcd319-9564-451b-ab9c-f96c01fe6bd2 service cinder] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:19:11.883450 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-c9bcd319-9564-451b-ab9c-f96c01fe6bd2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:19:11.940867 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bcd319-9564-451b-ab9c-f96c01fe6bd2 service cinder] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:19:11.944480 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28487898-4fba-4b95-a161-305e21c98480 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:19:11.945080 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28487898-4fba-4b95-a161-305e21c98480 service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:11.945355 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28487898-4fba-4b95-a161-305e21c98480 service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:11.945621 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28487898-4fba-4b95-a161-305e21c98480 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:11.947359 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-28487898-4fba-4b95-a161-305e21c98480 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:19:12.011135 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28487898-4fba-4b95-a161-305e21c98480 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:19:12.012596 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28487898-4fba-4b95-a161-305e21c98480 service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:19:12.021892 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1412/2840] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:19:11 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:19:14.844941 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=26 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:19:22.017686 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=50 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:19:29.695001 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5dfc7-8634-479c-89cd-0e40f649b527 service cinder] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:19:29.756892 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5dfc7-8634-479c-89cd-0e40f649b527 service cinder] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:19:29.762102 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b899f6-53fe-4b8d-9a30-e2002a39d515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:19:29.762757 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b899f6-53fe-4b8d-9a30-e2002a39d515 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:29.763035 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b899f6-53fe-4b8d-9a30-e2002a39d515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:29.763351 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b899f6-53fe-4b8d-9a30-e2002a39d515 admin admin] PATH_INFO: `/v3/users/f6d2b34ebaaf4e578dea262d52989e07` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:29.764476 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b899f6-53fe-4b8d-9a30-e2002a39d515 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6d2b34ebaaf4e578dea262d52989e07)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:19:29.766418 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b899f6-53fe-4b8d-9a30-e2002a39d515 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:19:29.812218 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-05b899f6-53fe-4b8d-9a30-e2002a39d515 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6d2b34ebaaf4e578dea262d52989e07'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:29.812862 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-05b899f6-53fe-4b8d-9a30-e2002a39d515 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6d2b34ebaaf4e578dea262d52989e07'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:29.820771 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-05b899f6-53fe-4b8d-9a30-e2002a39d515 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6d2b34ebaaf4e578dea262d52989e07'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:29.828076 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-05b899f6-53fe-4b8d-9a30-e2002a39d515 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6d2b34ebaaf4e578dea262d52989e07'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:29.828733 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-05b899f6-53fe-4b8d-9a30-e2002a39d515 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6d2b34ebaaf4e578dea262d52989e07'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:29.847347 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1429/2841] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 15:19:29 2022] DELETE /identity/v3/users/f6d2b34ebaaf4e578dea262d52989e07 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:19:29.852230 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687440f4-4cf2-46dc-8798-18975121a5b5 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:29.852697 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687440f4-4cf2-46dc-8798-18975121a5b5 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:29.852814 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687440f4-4cf2-46dc-8798-18975121a5b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:29.872246 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-687440f4-4cf2-46dc-8798-18975121a5b5 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:19:29.873503 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-687440f4-4cf2-46dc-8798-18975121a5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:19:29.932768 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1413/2842] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:19:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:19:29.942366 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b899f6-53fe-4b8d-9a30-e2002a39d515 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:19:29.943771 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-05b899f6-53fe-4b8d-9a30-e2002a39d515 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:19:30.001684 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b899f6-53fe-4b8d-9a30-e2002a39d515 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:19:30.005586 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89791d3f-aeba-4fa1-82ef-dcdaccbcd2f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:19:30.006207 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89791d3f-aeba-4fa1-82ef-dcdaccbcd2f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:30.006509 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89791d3f-aeba-4fa1-82ef-dcdaccbcd2f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:30.006780 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89791d3f-aeba-4fa1-82ef-dcdaccbcd2f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:30.016872 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89791d3f-aeba-4fa1-82ef-dcdaccbcd2f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:19:30.018516 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89791d3f-aeba-4fa1-82ef-dcdaccbcd2f2 service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:19:30.028780 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1430/2843] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:19:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:19:30.296648 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687440f4-4cf2-46dc-8798-18975121a5b5 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:19:30.298189 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-687440f4-4cf2-46dc-8798-18975121a5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:19:30.310487 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687440f4-4cf2-46dc-8798-18975121a5b5 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:19:30.315607 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dec335e-4399-4133-9858-333397414ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:19:30.316316 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec335e-4399-4133-9858-333397414ea0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:30.316655 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec335e-4399-4133-9858-333397414ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:30.316991 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec335e-4399-4133-9858-333397414ea0 admin admin] PATH_INFO: `/v3/projects/49ebb168eca54a0fac401af8f3e4e46e` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:30.318523 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dec335e-4399-4133-9858-333397414ea0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49ebb168eca54a0fac401af8f3e4e46e)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:19:30.320470 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dec335e-4399-4133-9858-333397414ea0 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:19:30.369588 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-7dec335e-4399-4133-9858-333397414ea0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49ebb168eca54a0fac401af8f3e4e46e', 'request_id': 'req-7dec335e-4399-4133-9858-333397414ea0'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:30.375269 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-7dec335e-4399-4133-9858-333397414ea0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49ebb168eca54a0fac401af8f3e4e46e', 'request_id': 'req-7dec335e-4399-4133-9858-333397414ea0'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:30.381389 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-7dec335e-4399-4133-9858-333397414ea0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49ebb168eca54a0fac401af8f3e4e46e', 'request_id': 'req-7dec335e-4399-4133-9858-333397414ea0'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:30.394189 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-7dec335e-4399-4133-9858-333397414ea0 admin admin] The token cache is being invalidate because project 49ebb168eca54a0fac401af8f3e4e46e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:19:30.394478 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-7dec335e-4399-4133-9858-333397414ea0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:30.395110 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-7dec335e-4399-4133-9858-333397414ea0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:30.396643 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1414/2844] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 15:19:30 2022] DELETE /identity/v3/projects/49ebb168eca54a0fac401af8f3e4e46e => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:19:30.401216 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89791d3f-aeba-4fa1-82ef-dcdaccbcd2f2 service neutron] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:19:30.410788 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89791d3f-aeba-4fa1-82ef-dcdaccbcd2f2 service neutron] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:19:30.428643 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:19:30.470146 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1f3fb8-38e1-4c6e-8d3b-fe8a09ddedcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:19:30.470700 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1f3fb8-38e1-4c6e-8d3b-fe8a09ddedcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:30.470975 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1f3fb8-38e1-4c6e-8d3b-fe8a09ddedcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:30.471275 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1f3fb8-38e1-4c6e-8d3b-fe8a09ddedcc admin admin] PATH_INFO: `/v3/users/3ce4293863dd4ec0b3d4198ea7aa639d` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:30.472406 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1f3fb8-38e1-4c6e-8d3b-fe8a09ddedcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ce4293863dd4ec0b3d4198ea7aa639d)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:19:30.474308 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1f3fb8-38e1-4c6e-8d3b-fe8a09ddedcc admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:19:30.519196 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-dc1f3fb8-38e1-4c6e-8d3b-fe8a09ddedcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ce4293863dd4ec0b3d4198ea7aa639d'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:30.519886 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-dc1f3fb8-38e1-4c6e-8d3b-fe8a09ddedcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ce4293863dd4ec0b3d4198ea7aa639d'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:30.527853 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-dc1f3fb8-38e1-4c6e-8d3b-fe8a09ddedcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ce4293863dd4ec0b3d4198ea7aa639d'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:30.534617 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-dc1f3fb8-38e1-4c6e-8d3b-fe8a09ddedcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ce4293863dd4ec0b3d4198ea7aa639d'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:30.535291 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-dc1f3fb8-38e1-4c6e-8d3b-fe8a09ddedcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ce4293863dd4ec0b3d4198ea7aa639d'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:30.552154 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1431/2845] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 15:19:30 2022] DELETE /identity/v3/users/3ce4293863dd4ec0b3d4198ea7aa639d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:19:30.818593 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dec335e-4399-4133-9858-333397414ea0 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:19:30.820411 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-7dec335e-4399-4133-9858-333397414ea0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:19:30.834661 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dec335e-4399-4133-9858-333397414ea0 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:19:30.892416 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e12d68-387f-4747-8212-ece614b4a9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:19:30.893076 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e12d68-387f-4747-8212-ece614b4a9f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:30.893361 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e12d68-387f-4747-8212-ece614b4a9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:30.893632 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e12d68-387f-4747-8212-ece614b4a9f0 admin admin] PATH_INFO: `/v3/projects/24879e909603453a84d215121b0f73cb` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:30.894742 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e12d68-387f-4747-8212-ece614b4a9f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24879e909603453a84d215121b0f73cb)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:19:30.896692 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e12d68-387f-4747-8212-ece614b4a9f0 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:19:30.926933 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:19:30.944866 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-38e12d68-387f-4747-8212-ece614b4a9f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24879e909603453a84d215121b0f73cb', 'request_id': 'req-38e12d68-387f-4747-8212-ece614b4a9f0'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:30.950547 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-38e12d68-387f-4747-8212-ece614b4a9f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24879e909603453a84d215121b0f73cb', 'request_id': 'req-38e12d68-387f-4747-8212-ece614b4a9f0'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:30.956667 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-38e12d68-387f-4747-8212-ece614b4a9f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24879e909603453a84d215121b0f73cb', 'request_id': 'req-38e12d68-387f-4747-8212-ece614b4a9f0'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:30.968838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-38e12d68-387f-4747-8212-ece614b4a9f0 admin admin] The token cache is being invalidate because project 24879e909603453a84d215121b0f73cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:19:30.969124 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-38e12d68-387f-4747-8212-ece614b4a9f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:30.969769 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-38e12d68-387f-4747-8212-ece614b4a9f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:30.971276 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1415/2846] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 15:19:30 2022] DELETE /identity/v3/projects/24879e909603453a84d215121b0f73cb => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 15:19:30.976304 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c974ed-1a36-4bf7-ae3c-d6e538fe9b20 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:30.976742 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c974ed-1a36-4bf7-ae3c-d6e538fe9b20 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:30.976848 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c974ed-1a36-4bf7-ae3c-d6e538fe9b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:30.997845 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-42c974ed-1a36-4bf7-ae3c-d6e538fe9b20 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:19:30.998940 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-42c974ed-1a36-4bf7-ae3c-d6e538fe9b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:19:31.056260 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1432/2847] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:19:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:19:31.064781 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1761fd9f-6e60-4b3c-bdae-b4d754aa0969 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:31.065068 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1761fd9f-6e60-4b3c-bdae-b4d754aa0969 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:31.065345 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1761fd9f-6e60-4b3c-bdae-b4d754aa0969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:31.085069 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-1761fd9f-6e60-4b3c-bdae-b4d754aa0969 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:19:31.086308 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-1761fd9f-6e60-4b3c-bdae-b4d754aa0969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:19:31.094475 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1416/2848] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:19:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:19:31.098684 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c974ed-1a36-4bf7-ae3c-d6e538fe9b20 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:19:31.109116 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c974ed-1a36-4bf7-ae3c-d6e538fe9b20 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:19:31.113594 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9831c242-7f9b-4903-bef9-d69ac5c1e297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:19:31.114184 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831c242-7f9b-4903-bef9-d69ac5c1e297 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:31.114459 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831c242-7f9b-4903-bef9-d69ac5c1e297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:31.114725 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831c242-7f9b-4903-bef9-d69ac5c1e297 admin admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:31.115401 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9831c242-7f9b-4903-bef9-d69ac5c1e297 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:19:31.115698 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9831c242-7f9b-4903-bef9-d69ac5c1e297 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:19:31.117258 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9831c242-7f9b-4903-bef9-d69ac5c1e297 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:19:31.129516 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1433/2849] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 15:19:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:19:31.133724 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1761fd9f-6e60-4b3c-bdae-b4d754aa0969 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:19:31.143116 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1761fd9f-6e60-4b3c-bdae-b4d754aa0969 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:19:31.148058 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fec748-f5db-4bec-a273-10c75539b70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:19:31.148651 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fec748-f5db-4bec-a273-10c75539b70d admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:31.148935 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fec748-f5db-4bec-a273-10c75539b70d admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:31.149208 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fec748-f5db-4bec-a273-10c75539b70d admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:31.149836 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fec748-f5db-4bec-a273-10c75539b70d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:19:31.151845 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fec748-f5db-4bec-a273-10c75539b70d admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:19:31.179007 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1417/2850] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 15:19:31 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:19:31.184081 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9831c242-7f9b-4903-bef9-d69ac5c1e297 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:19:31.194003 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9831c242-7f9b-4903-bef9-d69ac5c1e297 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:19:31.220121 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:19:31.252430 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c122795-c618-4a3a-b5e9-52d2626d98ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:19:31.252957 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c122795-c618-4a3a-b5e9-52d2626d98ba admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:31.253230 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c122795-c618-4a3a-b5e9-52d2626d98ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:31.253495 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c122795-c618-4a3a-b5e9-52d2626d98ba admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:31.254114 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c122795-c618-4a3a-b5e9-52d2626d98ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:19:31.256041 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c122795-c618-4a3a-b5e9-52d2626d98ba admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:19:31.276980 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1434/2851] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 15:19:31 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:19:31.281138 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fec748-f5db-4bec-a273-10c75539b70d admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:19:31.291933 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fec748-f5db-4bec-a273-10c75539b70d admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:19:31.296623 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae301415-33b0-4dff-a4e0-07de08c3708e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:19:31.297211 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae301415-33b0-4dff-a4e0-07de08c3708e admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:31.297493 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae301415-33b0-4dff-a4e0-07de08c3708e admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:31.297764 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae301415-33b0-4dff-a4e0-07de08c3708e admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:31.298421 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae301415-33b0-4dff-a4e0-07de08c3708e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:19:31.300031 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae301415-33b0-4dff-a4e0-07de08c3708e admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:19:31.309921 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1418/2852] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:19:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:19:31.313701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c122795-c618-4a3a-b5e9-52d2626d98ba admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:19:31.323546 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c122795-c618-4a3a-b5e9-52d2626d98ba admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:19:31.328405 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a3bc3d-016a-42eb-ab50-2560bf671f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:19:31.329019 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a3bc3d-016a-42eb-ab50-2560bf671f8a admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:31.329299 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a3bc3d-016a-42eb-ab50-2560bf671f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:31.329568 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a3bc3d-016a-42eb-ab50-2560bf671f8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:31.330197 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a3bc3d-016a-42eb-ab50-2560bf671f8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:19:31.331794 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a3bc3d-016a-42eb-ab50-2560bf671f8a admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:19:31.342993 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1435/2853] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:19:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:19:31.347019 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae301415-33b0-4dff-a4e0-07de08c3708e admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:19:31.358039 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae301415-33b0-4dff-a4e0-07de08c3708e admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:19:31.362658 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbc0474-0d79-4d67-bb40-9b82d4295e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:19:31.363333 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc0474-0d79-4d67-bb40-9b82d4295e9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:31.363618 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc0474-0d79-4d67-bb40-9b82d4295e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:31.363891 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc0474-0d79-4d67-bb40-9b82d4295e9c admin admin] PATH_INFO: `/v3/projects/87df9b1eed6f446aa66f47a8548c12a8/users/bf99a32b7a834adda04674592870b308/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:31.379502 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbc0474-0d79-4d67-bb40-9b82d4295e9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=87df9b1eed6f446aa66f47a8548c12a8, user_id=bf99a32b7a834adda04674592870b308, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:19:31.382146 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbc0474-0d79-4d67-bb40-9b82d4295e9c admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:19:31.395398 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1419/2854] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 15:19:31 2022] PUT /identity/v3/projects/87df9b1eed6f446aa66f47a8548c12a8/users/bf99a32b7a834adda04674592870b308/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:19:31.407345 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d0ef4c-dab8-4535-919e-5de0c7dfd88a None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:31.407631 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d0ef4c-dab8-4535-919e-5de0c7dfd88a None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:31.407904 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d0ef4c-dab8-4535-919e-5de0c7dfd88a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:31.437499 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-03d0ef4c-dab8-4535-919e-5de0c7dfd88a None None] MFA Rules not processed for user `bf99a32b7a834adda04674592870b308`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:19:31.438699 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-03d0ef4c-dab8-4535-919e-5de0c7dfd88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:19:31.506205 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1436/2855] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:19:31 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:19:31.510993 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbc0474-0d79-4d67-bb40-9b82d4295e9c admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:19:31.572838 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbc0474-0d79-4d67-bb40-9b82d4295e9c admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:19:31.578885 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081449cf-a774-45cc-b1f7-f4e823a23b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:19:31.579525 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081449cf-a774-45cc-b1f7-f4e823a23b1b admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:31.579807 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081449cf-a774-45cc-b1f7-f4e823a23b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:31.580081 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081449cf-a774-45cc-b1f7-f4e823a23b1b admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:31.580723 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081449cf-a774-45cc-b1f7-f4e823a23b1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:19:31.582676 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081449cf-a774-45cc-b1f7-f4e823a23b1b admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:19:31.609910 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1420/2856] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 15:19:31 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:19:31.614409 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d0ef4c-dab8-4535-919e-5de0c7dfd88a None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:19:31.626271 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d0ef4c-dab8-4535-919e-5de0c7dfd88a None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:19:31.682195 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2851d-7a7a-484c-9de9-0d740704a561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:19:31.682808 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2851d-7a7a-484c-9de9-0d740704a561 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:31.683072 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2851d-7a7a-484c-9de9-0d740704a561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:31.683381 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2851d-7a7a-484c-9de9-0d740704a561 admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:31.684025 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f2851d-7a7a-484c-9de9-0d740704a561 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:19:31.685930 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f2851d-7a7a-484c-9de9-0d740704a561 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:19:31.708341 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1437/2857] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 15:19:31 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:19:31.712343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081449cf-a774-45cc-b1f7-f4e823a23b1b admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:19:31.723294 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081449cf-a774-45cc-b1f7-f4e823a23b1b admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:19:31.728078 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327047be-085f-4fbe-9f04-1cc268eef841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:19:31.728773 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327047be-085f-4fbe-9f04-1cc268eef841 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:31.729056 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327047be-085f-4fbe-9f04-1cc268eef841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:31.729329 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327047be-085f-4fbe-9f04-1cc268eef841 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:31.729969 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327047be-085f-4fbe-9f04-1cc268eef841 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:19:31.731595 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327047be-085f-4fbe-9f04-1cc268eef841 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:19:31.742034 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1421/2858] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:19:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:19:31.745896 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2851d-7a7a-484c-9de9-0d740704a561 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:19:31.755990 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2851d-7a7a-484c-9de9-0d740704a561 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:19:31.761546 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2200df-10d0-4b06-8cb0-99db9e800299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:19:31.762124 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2200df-10d0-4b06-8cb0-99db9e800299 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:31.762439 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2200df-10d0-4b06-8cb0-99db9e800299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:31.762708 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2200df-10d0-4b06-8cb0-99db9e800299 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:31.763376 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2200df-10d0-4b06-8cb0-99db9e800299 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:19:31.765066 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2200df-10d0-4b06-8cb0-99db9e800299 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:19:31.775178 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1438/2859] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:19:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:19:31.780011 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327047be-085f-4fbe-9f04-1cc268eef841 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:19:31.789993 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327047be-085f-4fbe-9f04-1cc268eef841 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:19:31.795312 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b15a7b-fc35-4403-8e95-3c1f08a1d4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:19:31.795955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b15a7b-fc35-4403-8e95-3c1f08a1d4c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:31.796243 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b15a7b-fc35-4403-8e95-3c1f08a1d4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:31.796520 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b15a7b-fc35-4403-8e95-3c1f08a1d4c6 admin admin] PATH_INFO: `/v3/projects/a7f8c5f250b7497c8c350419b7e88575/users/3138d07832584c65ba33b571da77718b/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:31.810882 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b15a7b-fc35-4403-8e95-3c1f08a1d4c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7f8c5f250b7497c8c350419b7e88575, user_id=3138d07832584c65ba33b571da77718b, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:19:31.813564 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b15a7b-fc35-4403-8e95-3c1f08a1d4c6 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:19:31.821972 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:19:31.827909 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1422/2860] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 15:19:31 2022] PUT /identity/v3/projects/a7f8c5f250b7497c8c350419b7e88575/users/3138d07832584c65ba33b571da77718b/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:19:31.837859 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1fa1a7-2589-4b87-8912-b01fda116898 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:31.838712 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1fa1a7-2589-4b87-8912-b01fda116898 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:31.839041 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1fa1a7-2589-4b87-8912-b01fda116898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:31.866309 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-0e1fa1a7-2589-4b87-8912-b01fda116898 None None] MFA Rules not processed for user `3138d07832584c65ba33b571da77718b`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:19:31.867552 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-0e1fa1a7-2589-4b87-8912-b01fda116898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:19:31.871633 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:19:31.936520 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1439/2861] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:19:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:19:31.940978 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b15a7b-fc35-4403-8e95-3c1f08a1d4c6 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:19:32.002666 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b15a7b-fc35-4403-8e95-3c1f08a1d4c6 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:19:32.007341 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b956fc27-f868-4247-aafd-f72283069133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:19:32.007944 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b956fc27-f868-4247-aafd-f72283069133 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:32.008224 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b956fc27-f868-4247-aafd-f72283069133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:32.008502 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b956fc27-f868-4247-aafd-f72283069133 admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:32.009136 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b956fc27-f868-4247-aafd-f72283069133 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:19:32.011667 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b956fc27-f868-4247-aafd-f72283069133 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:19:32.037782 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1423/2862] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 15:19:31 2022] POST /identity/v3/projects => generated 387 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:19:32.041868 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1fa1a7-2589-4b87-8912-b01fda116898 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:19:32.052807 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1fa1a7-2589-4b87-8912-b01fda116898 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:19:32.108768 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5f21ff-3afc-4558-9b68-096dd782b8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:19:32.109359 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5f21ff-3afc-4558-9b68-096dd782b8ef admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:32.109639 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5f21ff-3afc-4558-9b68-096dd782b8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:32.109907 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5f21ff-3afc-4558-9b68-096dd782b8ef admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:32.110536 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5f21ff-3afc-4558-9b68-096dd782b8ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:19:32.112466 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5f21ff-3afc-4558-9b68-096dd782b8ef admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:19:32.134176 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1440/2863] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 15:19:32 2022] POST /identity/v3/users => generated 300 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:19:32.138677 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b956fc27-f868-4247-aafd-f72283069133 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:19:32.148936 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b956fc27-f868-4247-aafd-f72283069133 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:19:32.153669 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b46c12-6dd7-4542-b254-b9b17413b62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:19:32.154263 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b46c12-6dd7-4542-b254-b9b17413b62c admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:32.154544 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b46c12-6dd7-4542-b254-b9b17413b62c admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:32.154818 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b46c12-6dd7-4542-b254-b9b17413b62c admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:32.155484 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b46c12-6dd7-4542-b254-b9b17413b62c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:19:32.157078 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b46c12-6dd7-4542-b254-b9b17413b62c admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:19:32.167491 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1424/2864] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:19:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:19:32.171523 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5f21ff-3afc-4558-9b68-096dd782b8ef admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:19:32.181272 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5f21ff-3afc-4558-9b68-096dd782b8ef admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:19:32.186452 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c913746b-df67-488b-aa0e-ce5b771ef5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:19:32.187089 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c913746b-df67-488b-aa0e-ce5b771ef5ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:32.187410 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c913746b-df67-488b-aa0e-ce5b771ef5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:32.187684 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c913746b-df67-488b-aa0e-ce5b771ef5ad admin admin] PATH_INFO: `/v3/projects/ff9e7ad4c6af49288b7417b0080442ab/users/0d6d1fc80c0b48c8ae35007b918f1351/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:32.202977 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c913746b-df67-488b-aa0e-ce5b771ef5ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff9e7ad4c6af49288b7417b0080442ab, user_id=0d6d1fc80c0b48c8ae35007b918f1351, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:19:32.205697 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c913746b-df67-488b-aa0e-ce5b771ef5ad admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:19:32.219045 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1441/2865] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 15:19:32 2022] PUT /identity/v3/projects/ff9e7ad4c6af49288b7417b0080442ab/users/0d6d1fc80c0b48c8ae35007b918f1351/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:19:32.223383 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b46c12-6dd7-4542-b254-b9b17413b62c admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:19:32.233768 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b46c12-6dd7-4542-b254-b9b17413b62c admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:19:32.289514 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06edb1cb-0d1c-46f5-9bf4-acf94b10c462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:19:32.290032 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06edb1cb-0d1c-46f5-9bf4-acf94b10c462 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:32.290309 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06edb1cb-0d1c-46f5-9bf4-acf94b10c462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:32.290576 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06edb1cb-0d1c-46f5-9bf4-acf94b10c462 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:32.291247 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06edb1cb-0d1c-46f5-9bf4-acf94b10c462 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:19:32.292829 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06edb1cb-0d1c-46f5-9bf4-acf94b10c462 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:19:32.302199 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1425/2866] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:19:32 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:19:32.306269 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c913746b-df67-488b-aa0e-ce5b771ef5ad admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:19:32.316938 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c913746b-df67-488b-aa0e-ce5b771ef5ad admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:19:32.321473 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba1e939-d395-40fb-b7f8-077c6520b096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:19:32.322104 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1e939-d395-40fb-b7f8-077c6520b096 admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:32.323051 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1e939-d395-40fb-b7f8-077c6520b096 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:32.323353 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1e939-d395-40fb-b7f8-077c6520b096 admin admin] PATH_INFO: `/v3/projects/ff9e7ad4c6af49288b7417b0080442ab/users/0d6d1fc80c0b48c8ae35007b918f1351/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:32.325527 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba1e939-d395-40fb-b7f8-077c6520b096 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff9e7ad4c6af49288b7417b0080442ab, user_id=0d6d1fc80c0b48c8ae35007b918f1351, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:19:32.328173 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba1e939-d395-40fb-b7f8-077c6520b096 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:19:32.338139 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1442/2867] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 15:19:32 2022] PUT /identity/v3/projects/ff9e7ad4c6af49288b7417b0080442ab/users/0d6d1fc80c0b48c8ae35007b918f1351/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:19:32.348044 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9caaddf-cbb4-4aa2-a7b1-9a7858599373 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:32.348332 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9caaddf-cbb4-4aa2-a7b1-9a7858599373 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:32.348609 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9caaddf-cbb4-4aa2-a7b1-9a7858599373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:32.377192 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-c9caaddf-cbb4-4aa2-a7b1-9a7858599373 None None] MFA Rules not processed for user `0d6d1fc80c0b48c8ae35007b918f1351`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:19:32.378403 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-c9caaddf-cbb4-4aa2-a7b1-9a7858599373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:19:32.436859 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:19:32.447274 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1426/2868] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:19:32 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:19:32.458476 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba1e939-d395-40fb-b7f8-077c6520b096 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:19:32.460199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-6ba1e939-d395-40fb-b7f8-077c6520b096 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:19:32.517442 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba1e939-d395-40fb-b7f8-077c6520b096 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:19:32.521159 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4546b13-e972-4710-8fff-90cea55c865a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:19:32.521759 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4546b13-e972-4710-8fff-90cea55c865a service cinder] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:32.522040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4546b13-e972-4710-8fff-90cea55c865a service cinder] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:32.522309 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4546b13-e972-4710-8fff-90cea55c865a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:32.531187 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4546b13-e972-4710-8fff-90cea55c865a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:19:32.532632 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4546b13-e972-4710-8fff-90cea55c865a service cinder] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:19:32.537528 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:19:32.543492 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1443/2869] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:19:32 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:19:33.043189 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9caaddf-cbb4-4aa2-a7b1-9a7858599373 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:19:33.044610 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-c9caaddf-cbb4-4aa2-a7b1-9a7858599373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:19:33.056480 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9caaddf-cbb4-4aa2-a7b1-9a7858599373 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:19:33.060170 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5a3a37-623a-4dc7-99e2-06258f965721 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:19:33.060756 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a3a37-623a-4dc7-99e2-06258f965721 service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:33.061041 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a3a37-623a-4dc7-99e2-06258f965721 service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:33.061310 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a3a37-623a-4dc7-99e2-06258f965721 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:33.117876 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5a3a37-623a-4dc7-99e2-06258f965721 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:19:33.119495 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5a3a37-623a-4dc7-99e2-06258f965721 service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:19:33.128287 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1427/2870] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:19:33 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 15:19:37.132007 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4546b13-e972-4710-8fff-90cea55c865a service cinder] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:19:37.140793 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4546b13-e972-4710-8fff-90cea55c865a service cinder] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:19:37.144381 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc813970-99b0-463b-a1b4-45dcceb8a1a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:19:37.144966 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc813970-99b0-463b-a1b4-45dcceb8a1a4 service cinder] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:37.145238 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc813970-99b0-463b-a1b4-45dcceb8a1a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:37.145502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc813970-99b0-463b-a1b4-45dcceb8a1a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:37.206748 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc813970-99b0-463b-a1b4-45dcceb8a1a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:19:37.208155 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc813970-99b0-463b-a1b4-45dcceb8a1a4 service cinder] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:19:37.218337 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1444/2871] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:19:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:19:39.293131 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5a3a37-623a-4dc7-99e2-06258f965721 service cinder] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:19:39.355314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5a3a37-623a-4dc7-99e2-06258f965721 service cinder] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:19:39.359852 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03083ef8-e3ea-44bc-8544-24bc854ccf2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:19:39.360474 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03083ef8-e3ea-44bc-8544-24bc854ccf2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:39.360756 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03083ef8-e3ea-44bc-8544-24bc854ccf2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:39.361031 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03083ef8-e3ea-44bc-8544-24bc854ccf2b admin admin] PATH_INFO: `/v3/users/bf99a32b7a834adda04674592870b308` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:39.362143 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03083ef8-e3ea-44bc-8544-24bc854ccf2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf99a32b7a834adda04674592870b308)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:19:39.364095 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03083ef8-e3ea-44bc-8544-24bc854ccf2b admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:19:39.408721 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-03083ef8-e3ea-44bc-8544-24bc854ccf2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf99a32b7a834adda04674592870b308'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:39.409405 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-03083ef8-e3ea-44bc-8544-24bc854ccf2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf99a32b7a834adda04674592870b308'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:39.421978 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-03083ef8-e3ea-44bc-8544-24bc854ccf2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf99a32b7a834adda04674592870b308'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:39.428733 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-03083ef8-e3ea-44bc-8544-24bc854ccf2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf99a32b7a834adda04674592870b308'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:39.436095 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-03083ef8-e3ea-44bc-8544-24bc854ccf2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf99a32b7a834adda04674592870b308'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:39.441889 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1428/2872] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 15:19:39 2022] DELETE /identity/v3/users/bf99a32b7a834adda04674592870b308 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:19:39.446405 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db99ce-ffcb-448c-890b-6a523281bc21 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:39.446866 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db99ce-ffcb-448c-890b-6a523281bc21 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:39.446952 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db99ce-ffcb-448c-890b-6a523281bc21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:39.465993 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-c1db99ce-ffcb-448c-890b-6a523281bc21 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:19:39.467092 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-c1db99ce-ffcb-448c-890b-6a523281bc21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:19:39.524770 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1445/2873] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:19:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:19:39.535197 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03083ef8-e3ea-44bc-8544-24bc854ccf2b admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:19:39.536610 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-03083ef8-e3ea-44bc-8544-24bc854ccf2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:19:39.551868 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:19:39.598342 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03083ef8-e3ea-44bc-8544-24bc854ccf2b admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:19:39.602058 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10874671-4344-4699-8508-0e8913804121 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:19:39.602669 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10874671-4344-4699-8508-0e8913804121 service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:39.602951 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10874671-4344-4699-8508-0e8913804121 service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:39.603273 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10874671-4344-4699-8508-0e8913804121 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:39.612816 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10874671-4344-4699-8508-0e8913804121 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:19:39.614250 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10874671-4344-4699-8508-0e8913804121 service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:19:39.624299 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1429/2874] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:19:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:19:39.886966 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1db99ce-ffcb-448c-890b-6a523281bc21 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:19:39.888451 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-c1db99ce-ffcb-448c-890b-6a523281bc21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:19:39.900509 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1db99ce-ffcb-448c-890b-6a523281bc21 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:19:39.905225 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e3ce52-802e-40b5-9e62-b6f8e525de41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:19:39.905860 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e3ce52-802e-40b5-9e62-b6f8e525de41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:39.906144 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e3ce52-802e-40b5-9e62-b6f8e525de41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:39.906411 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e3ce52-802e-40b5-9e62-b6f8e525de41 admin admin] PATH_INFO: `/v3/projects/87df9b1eed6f446aa66f47a8548c12a8` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:39.907583 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e3ce52-802e-40b5-9e62-b6f8e525de41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87df9b1eed6f446aa66f47a8548c12a8)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:19:39.909483 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e3ce52-802e-40b5-9e62-b6f8e525de41 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:19:39.958685 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-a9e3ce52-802e-40b5-9e62-b6f8e525de41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87df9b1eed6f446aa66f47a8548c12a8', 'request_id': 'req-a9e3ce52-802e-40b5-9e62-b6f8e525de41'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:39.964378 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-a9e3ce52-802e-40b5-9e62-b6f8e525de41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87df9b1eed6f446aa66f47a8548c12a8', 'request_id': 'req-a9e3ce52-802e-40b5-9e62-b6f8e525de41'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:39.970375 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-a9e3ce52-802e-40b5-9e62-b6f8e525de41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87df9b1eed6f446aa66f47a8548c12a8', 'request_id': 'req-a9e3ce52-802e-40b5-9e62-b6f8e525de41'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:39.983498 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-a9e3ce52-802e-40b5-9e62-b6f8e525de41 admin admin] The token cache is being invalidate because project 87df9b1eed6f446aa66f47a8548c12a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:19:39.983781 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-a9e3ce52-802e-40b5-9e62-b6f8e525de41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:39.984400 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-a9e3ce52-802e-40b5-9e62-b6f8e525de41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:39.985867 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1446/2875] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 15:19:39 2022] DELETE /identity/v3/projects/87df9b1eed6f446aa66f47a8548c12a8 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:19:39.989903 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10874671-4344-4699-8508-0e8913804121 service neutron] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:19:40.000011 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10874671-4344-4699-8508-0e8913804121 service neutron] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:19:40.057799 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f846906b-12d3-4978-b38d-ae3fd7f2619d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:19:40.058547 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f846906b-12d3-4978-b38d-ae3fd7f2619d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:40.058831 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f846906b-12d3-4978-b38d-ae3fd7f2619d admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:40.059103 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f846906b-12d3-4978-b38d-ae3fd7f2619d admin admin] PATH_INFO: `/v3/users/3138d07832584c65ba33b571da77718b` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:40.060268 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f846906b-12d3-4978-b38d-ae3fd7f2619d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3138d07832584c65ba33b571da77718b)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:19:40.062178 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f846906b-12d3-4978-b38d-ae3fd7f2619d admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:19:40.108093 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-f846906b-12d3-4978-b38d-ae3fd7f2619d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3138d07832584c65ba33b571da77718b'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:40.108754 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-f846906b-12d3-4978-b38d-ae3fd7f2619d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3138d07832584c65ba33b571da77718b'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:40.121304 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-f846906b-12d3-4978-b38d-ae3fd7f2619d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3138d07832584c65ba33b571da77718b'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:40.128379 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-f846906b-12d3-4978-b38d-ae3fd7f2619d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3138d07832584c65ba33b571da77718b'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:40.135545 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-f846906b-12d3-4978-b38d-ae3fd7f2619d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3138d07832584c65ba33b571da77718b'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:40.141437 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1430/2876] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 15:19:39 2022] DELETE /identity/v3/users/3138d07832584c65ba33b571da77718b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:19:40.405391 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e3ce52-802e-40b5-9e62-b6f8e525de41 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:19:40.406925 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-a9e3ce52-802e-40b5-9e62-b6f8e525de41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:19:40.422503 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e3ce52-802e-40b5-9e62-b6f8e525de41 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:19:40.442972 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:19:40.484948 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f49c88-893e-4e15-a60b-ea0a54e9dd5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:19:40.485610 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f49c88-893e-4e15-a60b-ea0a54e9dd5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:40.485853 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f49c88-893e-4e15-a60b-ea0a54e9dd5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:40.486123 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f49c88-893e-4e15-a60b-ea0a54e9dd5a admin admin] PATH_INFO: `/v3/projects/a7f8c5f250b7497c8c350419b7e88575` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:40.487260 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f49c88-893e-4e15-a60b-ea0a54e9dd5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7f8c5f250b7497c8c350419b7e88575)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:19:40.489156 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f49c88-893e-4e15-a60b-ea0a54e9dd5a admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:19:40.536816 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-b5f49c88-893e-4e15-a60b-ea0a54e9dd5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7f8c5f250b7497c8c350419b7e88575', 'request_id': 'req-b5f49c88-893e-4e15-a60b-ea0a54e9dd5a'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:40.542817 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-b5f49c88-893e-4e15-a60b-ea0a54e9dd5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7f8c5f250b7497c8c350419b7e88575', 'request_id': 'req-b5f49c88-893e-4e15-a60b-ea0a54e9dd5a'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:40.548926 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-b5f49c88-893e-4e15-a60b-ea0a54e9dd5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7f8c5f250b7497c8c350419b7e88575', 'request_id': 'req-b5f49c88-893e-4e15-a60b-ea0a54e9dd5a'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:40.562256 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-b5f49c88-893e-4e15-a60b-ea0a54e9dd5a admin admin] The token cache is being invalidate because project a7f8c5f250b7497c8c350419b7e88575 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:19:40.562560 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-b5f49c88-893e-4e15-a60b-ea0a54e9dd5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:40.563182 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-b5f49c88-893e-4e15-a60b-ea0a54e9dd5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:40.564748 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1447/2877] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 15:19:40 2022] DELETE /identity/v3/projects/a7f8c5f250b7497c8c350419b7e88575 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:19:40.568756 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f846906b-12d3-4978-b38d-ae3fd7f2619d admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:19:40.570216 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-f846906b-12d3-4978-b38d-ae3fd7f2619d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:19:40.584164 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f846906b-12d3-4978-b38d-ae3fd7f2619d admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:19:40.616992 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:19:40.642962 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f39891c-5fd1-4562-8f0d-28cd1d02b10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:19:40.643629 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39891c-5fd1-4562-8f0d-28cd1d02b10e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:40.643913 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39891c-5fd1-4562-8f0d-28cd1d02b10e admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:40.644185 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39891c-5fd1-4562-8f0d-28cd1d02b10e admin admin] PATH_INFO: `/v3/users/0d6d1fc80c0b48c8ae35007b918f1351` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:40.645304 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f39891c-5fd1-4562-8f0d-28cd1d02b10e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d6d1fc80c0b48c8ae35007b918f1351)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:19:40.647237 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f39891c-5fd1-4562-8f0d-28cd1d02b10e admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:19:40.691707 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-1f39891c-5fd1-4562-8f0d-28cd1d02b10e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d6d1fc80c0b48c8ae35007b918f1351'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:40.692348 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-1f39891c-5fd1-4562-8f0d-28cd1d02b10e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d6d1fc80c0b48c8ae35007b918f1351'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:40.704939 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-1f39891c-5fd1-4562-8f0d-28cd1d02b10e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d6d1fc80c0b48c8ae35007b918f1351'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:40.711725 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-1f39891c-5fd1-4562-8f0d-28cd1d02b10e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d6d1fc80c0b48c8ae35007b918f1351'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:40.719609 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-1f39891c-5fd1-4562-8f0d-28cd1d02b10e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d6d1fc80c0b48c8ae35007b918f1351'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:40.725532 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1431/2878] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 15:19:40 2022] DELETE /identity/v3/users/0d6d1fc80c0b48c8ae35007b918f1351 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:19:41.003929 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f49c88-893e-4e15-a60b-ea0a54e9dd5a admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:19:41.005346 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-b5f49c88-893e-4e15-a60b-ea0a54e9dd5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:19:41.020177 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f49c88-893e-4e15-a60b-ea0a54e9dd5a admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:19:41.085307 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af74a23-25d8-4e7c-91ee-6f697fb1e9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:19:41.085954 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af74a23-25d8-4e7c-91ee-6f697fb1e9f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:41.086694 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af74a23-25d8-4e7c-91ee-6f697fb1e9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:41.086694 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af74a23-25d8-4e7c-91ee-6f697fb1e9f1 admin admin] PATH_INFO: `/v3/projects/ff9e7ad4c6af49288b7417b0080442ab` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:41.087693 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af74a23-25d8-4e7c-91ee-6f697fb1e9f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff9e7ad4c6af49288b7417b0080442ab)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:19:41.089585 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af74a23-25d8-4e7c-91ee-6f697fb1e9f1 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:19:41.140559 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-6af74a23-25d8-4e7c-91ee-6f697fb1e9f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff9e7ad4c6af49288b7417b0080442ab', 'request_id': 'req-6af74a23-25d8-4e7c-91ee-6f697fb1e9f1'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:41.144963 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:19:41.147170 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-6af74a23-25d8-4e7c-91ee-6f697fb1e9f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff9e7ad4c6af49288b7417b0080442ab', 'request_id': 'req-6af74a23-25d8-4e7c-91ee-6f697fb1e9f1'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:41.156084 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-6af74a23-25d8-4e7c-91ee-6f697fb1e9f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff9e7ad4c6af49288b7417b0080442ab', 'request_id': 'req-6af74a23-25d8-4e7c-91ee-6f697fb1e9f1'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:41.170496 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-6af74a23-25d8-4e7c-91ee-6f697fb1e9f1 admin admin] The token cache is being invalidate because project ff9e7ad4c6af49288b7417b0080442ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:19:41.170792 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-6af74a23-25d8-4e7c-91ee-6f697fb1e9f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:41.171503 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-6af74a23-25d8-4e7c-91ee-6f697fb1e9f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:41.173199 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1448/2879] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 15:19:41 2022] DELETE /identity/v3/projects/ff9e7ad4c6af49288b7417b0080442ab => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:19:41.180349 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7382e64-e434-4188-ac51-f63351e0b771 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:41.180732 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7382e64-e434-4188-ac51-f63351e0b771 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:41.181071 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7382e64-e434-4188-ac51-f63351e0b771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:41.186398 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af74a23-25d8-4e7c-91ee-6f697fb1e9f1 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:19:41.187846 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-6af74a23-25d8-4e7c-91ee-6f697fb1e9f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:19:41.212772 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-e7382e64-e434-4188-ac51-f63351e0b771 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:19:41.214158 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-e7382e64-e434-4188-ac51-f63351e0b771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:19:41.252665 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af74a23-25d8-4e7c-91ee-6f697fb1e9f1 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:19:41.257040 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac97557-105b-418a-8c27-fe3fde27f058 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:19:41.257659 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac97557-105b-418a-8c27-fe3fde27f058 service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:41.257956 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac97557-105b-418a-8c27-fe3fde27f058 service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:41.258297 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac97557-105b-418a-8c27-fe3fde27f058 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:41.261165 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-5ac97557-105b-418a-8c27-fe3fde27f058 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:19:41.290111 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1432/2880] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:19:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:19:41.298317 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfcb364-adc0-4b96-9ade-1314e1e15179 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:41.298605 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfcb364-adc0-4b96-9ade-1314e1e15179 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:41.298871 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfcb364-adc0-4b96-9ade-1314e1e15179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:41.321530 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-8dfcb364-adc0-4b96-9ade-1314e1e15179 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:19:41.323332 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-8dfcb364-adc0-4b96-9ade-1314e1e15179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:19:41.331971 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1433/2881] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:19:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:19:41.336175 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfcb364-adc0-4b96-9ade-1314e1e15179 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:19:41.345665 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfcb364-adc0-4b96-9ade-1314e1e15179 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:19:41.349459 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac97557-105b-418a-8c27-fe3fde27f058 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:19:41.350453 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b5e9a5-db74-4be2-a27d-f27db0e25325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:19:41.351080 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5e9a5-db74-4be2-a27d-f27db0e25325 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:41.351396 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5e9a5-db74-4be2-a27d-f27db0e25325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:41.351667 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b5e9a5-db74-4be2-a27d-f27db0e25325 admin admin] PATH_INFO: `/v3/domains` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:41.351831 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac97557-105b-418a-8c27-fe3fde27f058 service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:19:41.352311 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b5e9a5-db74-4be2-a27d-f27db0e25325 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81689) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:19:41.352608 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b5e9a5-db74-4be2-a27d-f27db0e25325 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:19:41.354224 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b5e9a5-db74-4be2-a27d-f27db0e25325 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:19:41.366682 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1434/2882] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 15:19:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 15:19:41.367434 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1449/2883] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:19:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 15:19:41.371441 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b5e9a5-db74-4be2-a27d-f27db0e25325 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:19:41.381121 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b5e9a5-db74-4be2-a27d-f27db0e25325 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:19:41.386732 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8678d5a4-1872-4164-a396-14790612c470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:19:41.387359 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678d5a4-1872-4164-a396-14790612c470 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:41.387639 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678d5a4-1872-4164-a396-14790612c470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:41.387909 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678d5a4-1872-4164-a396-14790612c470 admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:41.388555 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8678d5a4-1872-4164-a396-14790612c470 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:19:41.390434 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8678d5a4-1872-4164-a396-14790612c470 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:19:41.418874 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1435/2884] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 15:19:41 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:19:41.423131 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac97557-105b-418a-8c27-fe3fde27f058 service neutron] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:19:41.435572 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac97557-105b-418a-8c27-fe3fde27f058 service neutron] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:19:41.501372 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d39d43-850f-4133-b6a0-afdd58995754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:19:41.501997 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d39d43-850f-4133-b6a0-afdd58995754 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:41.502242 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d39d43-850f-4133-b6a0-afdd58995754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:41.502527 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d39d43-850f-4133-b6a0-afdd58995754 admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:41.503137 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d39d43-850f-4133-b6a0-afdd58995754 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:19:41.505068 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d39d43-850f-4133-b6a0-afdd58995754 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:19:41.531508 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1450/2885] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 15:19:41 2022] POST /identity/v3/users => generated 289 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:19:41.532836 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8678d5a4-1872-4164-a396-14790612c470 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:19:41.545438 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8678d5a4-1872-4164-a396-14790612c470 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:19:41.550148 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc4ac1-dc37-4859-9e08-3e6b98fe9d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:19:41.550755 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc4ac1-dc37-4859-9e08-3e6b98fe9d4e admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:41.551170 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc4ac1-dc37-4859-9e08-3e6b98fe9d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:41.551486 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc4ac1-dc37-4859-9e08-3e6b98fe9d4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:41.552131 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc4ac1-dc37-4859-9e08-3e6b98fe9d4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:19:41.553766 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc4ac1-dc37-4859-9e08-3e6b98fe9d4e admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:19:41.568610 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1436/2886] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:19:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:19:41.573635 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d39d43-850f-4133-b6a0-afdd58995754 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:19:41.586812 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d39d43-850f-4133-b6a0-afdd58995754 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:19:41.593746 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e585b06-4a3b-41c2-8c78-9852a6629672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:19:41.594384 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e585b06-4a3b-41c2-8c78-9852a6629672 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:41.594794 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e585b06-4a3b-41c2-8c78-9852a6629672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:41.595098 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e585b06-4a3b-41c2-8c78-9852a6629672 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:41.595773 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e585b06-4a3b-41c2-8c78-9852a6629672 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:19:41.597352 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e585b06-4a3b-41c2-8c78-9852a6629672 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:19:41.608850 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1451/2887] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:19:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:19:41.613138 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc4ac1-dc37-4859-9e08-3e6b98fe9d4e admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:19:41.618387 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:19:41.625416 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc4ac1-dc37-4859-9e08-3e6b98fe9d4e admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:19:41.631381 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47cd3a0-e7c1-4383-b3ab-da4818012fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:19:41.631980 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47cd3a0-e7c1-4383-b3ab-da4818012fe8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:41.632256 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47cd3a0-e7c1-4383-b3ab-da4818012fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:41.632528 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47cd3a0-e7c1-4383-b3ab-da4818012fe8 admin admin] PATH_INFO: `/v3/projects/7fbd319be05f439e833b19bd15dc851d/users/fe111c61085f4f36bceb0af25584435f/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:41.647618 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47cd3a0-e7c1-4383-b3ab-da4818012fe8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fbd319be05f439e833b19bd15dc851d, user_id=fe111c61085f4f36bceb0af25584435f, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:19:41.650677 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47cd3a0-e7c1-4383-b3ab-da4818012fe8 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:19:41.664240 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1437/2888] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 15:19:41 2022] PUT /identity/v3/projects/7fbd319be05f439e833b19bd15dc851d/users/fe111c61085f4f36bceb0af25584435f/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:19:41.676179 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0dcb1-f16d-4990-a17c-d2a9207e5e61 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:41.676907 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0dcb1-f16d-4990-a17c-d2a9207e5e61 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:41.676907 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0dcb1-f16d-4990-a17c-d2a9207e5e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:41.684440 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:19:41.709008 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-0fc0dcb1-f16d-4990-a17c-d2a9207e5e61 None None] MFA Rules not processed for user `fe111c61085f4f36bceb0af25584435f`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:19:41.710224 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-0fc0dcb1-f16d-4990-a17c-d2a9207e5e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:19:41.778378 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1452/2889] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:19:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:19:41.790496 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47cd3a0-e7c1-4383-b3ab-da4818012fe8 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:19:41.793030 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-e47cd3a0-e7c1-4383-b3ab-da4818012fe8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:19:41.850037 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47cd3a0-e7c1-4383-b3ab-da4818012fe8 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:19:41.854616 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf12931-174b-4f0b-ac30-e7c25c228e41 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6889402e54424bbd8d42c7bb759cf700', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:19:41.855253 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf12931-174b-4f0b-ac30-e7c25c228e41 service glance] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:41.855535 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf12931-174b-4f0b-ac30-e7c25c228e41 service glance] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:41.855805 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf12931-174b-4f0b-ac30-e7c25c228e41 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:41.864199 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf12931-174b-4f0b-ac30-e7c25c228e41 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:19:41.865653 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf12931-174b-4f0b-ac30-e7c25c228e41 service glance] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:19:41.874942 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1438/2890] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:19:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 15:19:51.774628 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=31 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:19:51.870917 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=35,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:19:59.479483 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc0dcb1-f16d-4990-a17c-d2a9207e5e61 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:19:59.541851 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc0dcb1-f16d-4990-a17c-d2a9207e5e61 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:19:59.546324 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffda012-f30d-4cae-816a-3b590d04f077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:19:59.546948 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffda012-f30d-4cae-816a-3b590d04f077 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:59.547272 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffda012-f30d-4cae-816a-3b590d04f077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:59.547545 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffda012-f30d-4cae-816a-3b590d04f077 admin admin] PATH_INFO: `/v3/users/fe111c61085f4f36bceb0af25584435f` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:59.548663 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffda012-f30d-4cae-816a-3b590d04f077 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe111c61085f4f36bceb0af25584435f)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:19:59.550604 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffda012-f30d-4cae-816a-3b590d04f077 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:19:59.596425 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-cffda012-f30d-4cae-816a-3b590d04f077 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe111c61085f4f36bceb0af25584435f'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:59.597421 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-cffda012-f30d-4cae-816a-3b590d04f077 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe111c61085f4f36bceb0af25584435f'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:59.604632 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-cffda012-f30d-4cae-816a-3b590d04f077 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe111c61085f4f36bceb0af25584435f'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:59.612326 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-cffda012-f30d-4cae-816a-3b590d04f077 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe111c61085f4f36bceb0af25584435f'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:59.612326 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-cffda012-f30d-4cae-816a-3b590d04f077 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe111c61085f4f36bceb0af25584435f'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:19:59.630132 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1453/2891] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 15:19:59 2022] DELETE /identity/v3/users/fe111c61085f4f36bceb0af25584435f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:19:59.634625 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb3d79d-b6a5-4389-be70-f164af86166f None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:59.635087 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb3d79d-b6a5-4389-be70-f164af86166f None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:59.635184 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb3d79d-b6a5-4389-be70-f164af86166f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:59.654680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-ecb3d79d-b6a5-4389-be70-f164af86166f None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:19:59.655967 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-ecb3d79d-b6a5-4389-be70-f164af86166f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:19:59.714712 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1439/2892] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:19:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:19:59.724172 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffda012-f30d-4cae-816a-3b590d04f077 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:19:59.725571 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-cffda012-f30d-4cae-816a-3b590d04f077 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:19:59.784755 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffda012-f30d-4cae-816a-3b590d04f077 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:19:59.788386 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1742d8cf-6710-4dd2-a0ce-d9d25465771d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:19:59.788971 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742d8cf-6710-4dd2-a0ce-d9d25465771d service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:19:59.789246 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742d8cf-6710-4dd2-a0ce-d9d25465771d service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:19:59.789512 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742d8cf-6710-4dd2-a0ce-d9d25465771d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:19:59.799263 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1742d8cf-6710-4dd2-a0ce-d9d25465771d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:19:59.800685 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1742d8cf-6710-4dd2-a0ce-d9d25465771d service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:19:59.810229 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1454/2893] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:19:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:20:00.074852 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb3d79d-b6a5-4389-be70-f164af86166f None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:20:00.076322 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-ecb3d79d-b6a5-4389-be70-f164af86166f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:20:00.089247 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb3d79d-b6a5-4389-be70-f164af86166f None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:20:00.094008 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1001aadf-00eb-4d58-a52a-cf93b0dd84ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:20:00.094680 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1001aadf-00eb-4d58-a52a-cf93b0dd84ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:00.094985 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1001aadf-00eb-4d58-a52a-cf93b0dd84ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:00.095276 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1001aadf-00eb-4d58-a52a-cf93b0dd84ed admin admin] PATH_INFO: `/v3/projects/7fbd319be05f439e833b19bd15dc851d` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:00.096427 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1001aadf-00eb-4d58-a52a-cf93b0dd84ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fbd319be05f439e833b19bd15dc851d)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:20:00.098385 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1001aadf-00eb-4d58-a52a-cf93b0dd84ed admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:20:00.145425 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-1001aadf-00eb-4d58-a52a-cf93b0dd84ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fbd319be05f439e833b19bd15dc851d', 'request_id': 'req-1001aadf-00eb-4d58-a52a-cf93b0dd84ed'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:20:00.151501 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-1001aadf-00eb-4d58-a52a-cf93b0dd84ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fbd319be05f439e833b19bd15dc851d', 'request_id': 'req-1001aadf-00eb-4d58-a52a-cf93b0dd84ed'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:20:00.157933 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-1001aadf-00eb-4d58-a52a-cf93b0dd84ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fbd319be05f439e833b19bd15dc851d', 'request_id': 'req-1001aadf-00eb-4d58-a52a-cf93b0dd84ed'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:20:00.169732 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-1001aadf-00eb-4d58-a52a-cf93b0dd84ed admin admin] The token cache is being invalidate because project 7fbd319be05f439e833b19bd15dc851d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:20:00.170025 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-1001aadf-00eb-4d58-a52a-cf93b0dd84ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:20:00.170648 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-1001aadf-00eb-4d58-a52a-cf93b0dd84ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:20:00.172211 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1440/2894] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 15:20:00 2022] DELETE /identity/v3/projects/7fbd319be05f439e833b19bd15dc851d => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:20:00.177483 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26272dc8-caa5-4c3b-b038-314ac80eeebf None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:00.177768 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26272dc8-caa5-4c3b-b038-314ac80eeebf None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:00.178037 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26272dc8-caa5-4c3b-b038-314ac80eeebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:00.198374 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-26272dc8-caa5-4c3b-b038-314ac80eeebf None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:20:00.199858 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-26272dc8-caa5-4c3b-b038-314ac80eeebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:20:00.218176 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:20:00.262687 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1455/2895] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:20:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:20:00.271694 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da8074-a93c-4a02-ba26-aeaf2e1ba539 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:00.271983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da8074-a93c-4a02-ba26-aeaf2e1ba539 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:00.272260 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da8074-a93c-4a02-ba26-aeaf2e1ba539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:00.292812 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-c6da8074-a93c-4a02-ba26-aeaf2e1ba539 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:20:00.294029 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-c6da8074-a93c-4a02-ba26-aeaf2e1ba539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:20:00.301945 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1441/2896] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:20:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:20:00.306200 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26272dc8-caa5-4c3b-b038-314ac80eeebf None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:20:00.316098 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26272dc8-caa5-4c3b-b038-314ac80eeebf None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:20:00.321012 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec430816-e4aa-4ea1-aab5-cc751ce2aafb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:20:00.322119 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec430816-e4aa-4ea1-aab5-cc751ce2aafb admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:00.322387 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec430816-e4aa-4ea1-aab5-cc751ce2aafb admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:00.322649 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec430816-e4aa-4ea1-aab5-cc751ce2aafb admin admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:00.323326 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec430816-e4aa-4ea1-aab5-cc751ce2aafb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:20:00.323623 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec430816-e4aa-4ea1-aab5-cc751ce2aafb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:20:00.325485 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec430816-e4aa-4ea1-aab5-cc751ce2aafb admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:20:00.336852 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1456/2897] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 15:20:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:20:00.341951 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da8074-a93c-4a02-ba26-aeaf2e1ba539 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:20:00.351284 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da8074-a93c-4a02-ba26-aeaf2e1ba539 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:20:00.356023 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4528a8-c970-4de6-b525-2a2724ee3d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:20:00.356616 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4528a8-c970-4de6-b525-2a2724ee3d56 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:00.356900 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4528a8-c970-4de6-b525-2a2724ee3d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:00.357171 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4528a8-c970-4de6-b525-2a2724ee3d56 admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:00.357808 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4528a8-c970-4de6-b525-2a2724ee3d56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:20:00.359760 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4528a8-c970-4de6-b525-2a2724ee3d56 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:20:00.386746 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1442/2898] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 15:20:00 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:20:00.390894 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec430816-e4aa-4ea1-aab5-cc751ce2aafb admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:20:00.401492 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec430816-e4aa-4ea1-aab5-cc751ce2aafb admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:20:00.456661 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3a6e07-5b97-4c45-8cd2-2e695ace2852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:20:00.457262 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a6e07-5b97-4c45-8cd2-2e695ace2852 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:00.457541 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a6e07-5b97-4c45-8cd2-2e695ace2852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:00.457811 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a6e07-5b97-4c45-8cd2-2e695ace2852 admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:00.458431 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3a6e07-5b97-4c45-8cd2-2e695ace2852 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:20:00.460378 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3a6e07-5b97-4c45-8cd2-2e695ace2852 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:20:00.481749 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1457/2899] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 15:20:00 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:20:00.485919 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4528a8-c970-4de6-b525-2a2724ee3d56 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:20:00.496475 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4528a8-c970-4de6-b525-2a2724ee3d56 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:20:00.501337 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929765ca-eed1-412a-b69f-d477316a4c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:20:00.501940 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929765ca-eed1-412a-b69f-d477316a4c89 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:00.502217 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929765ca-eed1-412a-b69f-d477316a4c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:00.502486 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929765ca-eed1-412a-b69f-d477316a4c89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:00.503159 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929765ca-eed1-412a-b69f-d477316a4c89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:20:00.504903 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929765ca-eed1-412a-b69f-d477316a4c89 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:20:00.516829 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1443/2900] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:20:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:20:00.520761 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3a6e07-5b97-4c45-8cd2-2e695ace2852 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:20:00.531480 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3a6e07-5b97-4c45-8cd2-2e695ace2852 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:20:00.536089 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d20cf6-0e1b-4ed1-9c8e-b2852cdbb504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:20:00.536677 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d20cf6-0e1b-4ed1-9c8e-b2852cdbb504 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:00.536952 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d20cf6-0e1b-4ed1-9c8e-b2852cdbb504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:00.537215 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d20cf6-0e1b-4ed1-9c8e-b2852cdbb504 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:00.537841 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d20cf6-0e1b-4ed1-9c8e-b2852cdbb504 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:20:00.539584 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d20cf6-0e1b-4ed1-9c8e-b2852cdbb504 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:20:00.550098 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1458/2901] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:20:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:20:00.554860 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929765ca-eed1-412a-b69f-d477316a4c89 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:20:00.565843 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929765ca-eed1-412a-b69f-d477316a4c89 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:20:00.570697 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f5fe5-6ca8-4480-871e-fb92b836569a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:20:00.572414 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f5fe5-6ca8-4480-871e-fb92b836569a admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:00.572698 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f5fe5-6ca8-4480-871e-fb92b836569a admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:00.572976 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f5fe5-6ca8-4480-871e-fb92b836569a admin admin] PATH_INFO: `/v3/projects/1e28965f2eb2432bba654eb79caacb0a/users/9f9ffd6504ff4ff7bdc73ecbd27ffe7f/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:00.589632 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231f5fe5-6ca8-4480-871e-fb92b836569a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e28965f2eb2432bba654eb79caacb0a, user_id=9f9ffd6504ff4ff7bdc73ecbd27ffe7f, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:20:00.592297 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231f5fe5-6ca8-4480-871e-fb92b836569a admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:20:00.605475 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1444/2902] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 15:20:00 2022] PUT /identity/v3/projects/1e28965f2eb2432bba654eb79caacb0a/users/9f9ffd6504ff4ff7bdc73ecbd27ffe7f/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:20:00.615508 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317276a7-7061-4e4d-a5d6-022bc2751068 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:00.615820 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317276a7-7061-4e4d-a5d6-022bc2751068 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:00.616094 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317276a7-7061-4e4d-a5d6-022bc2751068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:00.644207 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-317276a7-7061-4e4d-a5d6-022bc2751068 None None] MFA Rules not processed for user `9f9ffd6504ff4ff7bdc73ecbd27ffe7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:20:00.645405 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-317276a7-7061-4e4d-a5d6-022bc2751068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:20:00.708980 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1459/2903] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:20:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:20:00.720490 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f5fe5-6ca8-4480-871e-fb92b836569a admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:20:00.722231 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-231f5fe5-6ca8-4480-871e-fb92b836569a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:20:00.744898 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:20:00.781181 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f5fe5-6ca8-4480-871e-fb92b836569a admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:20:00.784797 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb9d9cd-9c05-47c1-8dd1-7b9501452672 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:20:00.785402 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb9d9cd-9c05-47c1-8dd1-7b9501452672 service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:00.785685 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb9d9cd-9c05-47c1-8dd1-7b9501452672 service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:00.785960 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb9d9cd-9c05-47c1-8dd1-7b9501452672 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:00.794174 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb9d9cd-9c05-47c1-8dd1-7b9501452672 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:20:00.795646 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb9d9cd-9c05-47c1-8dd1-7b9501452672 service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:20:00.804467 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1445/2904] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:20:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:20:03.342826 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317276a7-7061-4e4d-a5d6-022bc2751068 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:20:03.362311 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:20:03.406803 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317276a7-7061-4e4d-a5d6-022bc2751068 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:20:03.411332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4382d3-c796-4547-b190-e9ddffc5e330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:20:03.411952 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4382d3-c796-4547-b190-e9ddffc5e330 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:03.412228 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4382d3-c796-4547-b190-e9ddffc5e330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:03.412496 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4382d3-c796-4547-b190-e9ddffc5e330 admin admin] PATH_INFO: `/v3/users/9f9ffd6504ff4ff7bdc73ecbd27ffe7f` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:03.413598 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4382d3-c796-4547-b190-e9ddffc5e330 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f9ffd6504ff4ff7bdc73ecbd27ffe7f)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:20:03.415559 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4382d3-c796-4547-b190-e9ddffc5e330 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:20:03.462617 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-ff4382d3-c796-4547-b190-e9ddffc5e330 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f9ffd6504ff4ff7bdc73ecbd27ffe7f'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:20:03.463169 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-ff4382d3-c796-4547-b190-e9ddffc5e330 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f9ffd6504ff4ff7bdc73ecbd27ffe7f'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:20:03.470411 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-ff4382d3-c796-4547-b190-e9ddffc5e330 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f9ffd6504ff4ff7bdc73ecbd27ffe7f'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:20:03.477839 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-ff4382d3-c796-4547-b190-e9ddffc5e330 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f9ffd6504ff4ff7bdc73ecbd27ffe7f'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:20:03.478924 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-ff4382d3-c796-4547-b190-e9ddffc5e330 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f9ffd6504ff4ff7bdc73ecbd27ffe7f'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:20:03.497150 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1460/2905] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 15:20:03 2022] DELETE /identity/v3/users/9f9ffd6504ff4ff7bdc73ecbd27ffe7f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:20:03.501488 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11068527-1ea2-40c9-98b1-00518e801e27 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:03.501977 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11068527-1ea2-40c9-98b1-00518e801e27 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:03.502065 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11068527-1ea2-40c9-98b1-00518e801e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:03.523146 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-11068527-1ea2-40c9-98b1-00518e801e27 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:20:03.524979 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-11068527-1ea2-40c9-98b1-00518e801e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:20:03.708584 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1446/2906] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:20:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:20:03.718323 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4382d3-c796-4547-b190-e9ddffc5e330 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:20:03.720050 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-ff4382d3-c796-4547-b190-e9ddffc5e330 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:20:03.779330 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4382d3-c796-4547-b190-e9ddffc5e330 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:20:03.782889 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53692cf-4249-49cc-8567-42c0b6abf7e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:20:03.783530 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53692cf-4249-49cc-8567-42c0b6abf7e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:03.783807 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53692cf-4249-49cc-8567-42c0b6abf7e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:03.784074 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53692cf-4249-49cc-8567-42c0b6abf7e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:03.793025 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53692cf-4249-49cc-8567-42c0b6abf7e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:20:03.794456 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53692cf-4249-49cc-8567-42c0b6abf7e0 service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:20:03.803684 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1461/2907] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:20:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:20:04.083784 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11068527-1ea2-40c9-98b1-00518e801e27 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:20:04.085239 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-11068527-1ea2-40c9-98b1-00518e801e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:20:04.097791 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11068527-1ea2-40c9-98b1-00518e801e27 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:20:04.102499 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5604fc3-d1f6-4c71-955d-dc2679cabed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:20:04.103650 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5604fc3-d1f6-4c71-955d-dc2679cabed6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:04.103937 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5604fc3-d1f6-4c71-955d-dc2679cabed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:04.104209 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5604fc3-d1f6-4c71-955d-dc2679cabed6 admin admin] PATH_INFO: `/v3/projects/1e28965f2eb2432bba654eb79caacb0a` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:04.105370 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5604fc3-d1f6-4c71-955d-dc2679cabed6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e28965f2eb2432bba654eb79caacb0a)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:20:04.107312 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5604fc3-d1f6-4c71-955d-dc2679cabed6 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:20:04.152913 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-c5604fc3-d1f6-4c71-955d-dc2679cabed6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e28965f2eb2432bba654eb79caacb0a', 'request_id': 'req-c5604fc3-d1f6-4c71-955d-dc2679cabed6'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:20:04.158179 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-c5604fc3-d1f6-4c71-955d-dc2679cabed6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e28965f2eb2432bba654eb79caacb0a', 'request_id': 'req-c5604fc3-d1f6-4c71-955d-dc2679cabed6'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:20:04.164550 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-c5604fc3-d1f6-4c71-955d-dc2679cabed6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e28965f2eb2432bba654eb79caacb0a', 'request_id': 'req-c5604fc3-d1f6-4c71-955d-dc2679cabed6'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:20:04.176806 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-c5604fc3-d1f6-4c71-955d-dc2679cabed6 admin admin] The token cache is being invalidate because project 1e28965f2eb2432bba654eb79caacb0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:20:04.177089 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-c5604fc3-d1f6-4c71-955d-dc2679cabed6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:20:04.177684 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-c5604fc3-d1f6-4c71-955d-dc2679cabed6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:20:04.179123 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1447/2908] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 15:20:04 2022] DELETE /identity/v3/projects/1e28965f2eb2432bba654eb79caacb0a => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:20:04.184854 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b696c8eb-c901-48a8-8508-09f1c114b203 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:04.185142 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b696c8eb-c901-48a8-8508-09f1c114b203 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:04.185411 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b696c8eb-c901-48a8-8508-09f1c114b203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:04.205419 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-b696c8eb-c901-48a8-8508-09f1c114b203 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:20:04.206651 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-b696c8eb-c901-48a8-8508-09f1c114b203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:20:04.241417 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:20:04.270043 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1462/2909] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:20:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 15:20:04.278456 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7266035-337c-40b3-b893-60b3d3a5aa12 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:04.278706 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7266035-337c-40b3-b893-60b3d3a5aa12 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:04.278977 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7266035-337c-40b3-b893-60b3d3a5aa12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:04.298601 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-c7266035-337c-40b3-b893-60b3d3a5aa12 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:20:04.299839 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-c7266035-337c-40b3-b893-60b3d3a5aa12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:20:04.308138 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1448/2910] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:20:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:20:04.312280 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b696c8eb-c901-48a8-8508-09f1c114b203 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:20:04.322255 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b696c8eb-c901-48a8-8508-09f1c114b203 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:20:04.327152 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f987130-213d-4c38-a393-4f433ebc542e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:20:04.327786 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f987130-213d-4c38-a393-4f433ebc542e admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:04.328068 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f987130-213d-4c38-a393-4f433ebc542e admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:04.328345 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f987130-213d-4c38-a393-4f433ebc542e admin admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:04.328990 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f987130-213d-4c38-a393-4f433ebc542e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:20:04.329293 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f987130-213d-4c38-a393-4f433ebc542e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:20:04.330882 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f987130-213d-4c38-a393-4f433ebc542e admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:20:04.343039 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1463/2911] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 15:20:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:20:04.347354 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7266035-337c-40b3-b893-60b3d3a5aa12 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:20:04.357055 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7266035-337c-40b3-b893-60b3d3a5aa12 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:20:04.361710 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bdcaba-a0b4-4e92-bef2-af6396b3eced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:20:04.362302 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdcaba-a0b4-4e92-bef2-af6396b3eced admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:04.362582 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdcaba-a0b4-4e92-bef2-af6396b3eced admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:04.362854 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdcaba-a0b4-4e92-bef2-af6396b3eced admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:04.363526 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bdcaba-a0b4-4e92-bef2-af6396b3eced admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:20:04.365403 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bdcaba-a0b4-4e92-bef2-af6396b3eced admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:20:04.391745 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1449/2912] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 15:20:04 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:20:04.395890 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f987130-213d-4c38-a393-4f433ebc542e admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:20:04.406381 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f987130-213d-4c38-a393-4f433ebc542e admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:20:04.464579 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43647327-2b82-4b57-9075-45adaa38c670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:20:04.465180 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43647327-2b82-4b57-9075-45adaa38c670 admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:04.465460 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43647327-2b82-4b57-9075-45adaa38c670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:04.465733 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43647327-2b82-4b57-9075-45adaa38c670 admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:04.466369 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43647327-2b82-4b57-9075-45adaa38c670 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:20:04.468310 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43647327-2b82-4b57-9075-45adaa38c670 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:20:04.489539 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1464/2913] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 15:20:04 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:20:04.493622 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bdcaba-a0b4-4e92-bef2-af6396b3eced admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:20:04.503695 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bdcaba-a0b4-4e92-bef2-af6396b3eced admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:20:04.508239 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e6063-7dd4-4c7d-af23-a23c1fdd736b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:20:04.508819 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e6063-7dd4-4c7d-af23-a23c1fdd736b admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:04.509093 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e6063-7dd4-4c7d-af23-a23c1fdd736b admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:04.509368 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e6063-7dd4-4c7d-af23-a23c1fdd736b admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:04.510140 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972e6063-7dd4-4c7d-af23-a23c1fdd736b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:20:04.511744 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972e6063-7dd4-4c7d-af23-a23c1fdd736b admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:20:04.521705 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1450/2914] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:20:04 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:20:04.525808 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43647327-2b82-4b57-9075-45adaa38c670 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:20:04.535814 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43647327-2b82-4b57-9075-45adaa38c670 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:20:04.541301 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7702a643-3361-4bfb-97d9-85b82142586a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:20:04.541902 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702a643-3361-4bfb-97d9-85b82142586a admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:04.542186 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702a643-3361-4bfb-97d9-85b82142586a admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:04.542461 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702a643-3361-4bfb-97d9-85b82142586a admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:04.543139 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7702a643-3361-4bfb-97d9-85b82142586a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:20:04.544759 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7702a643-3361-4bfb-97d9-85b82142586a admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:20:04.555263 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1465/2915] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:20:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:20:04.559716 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e6063-7dd4-4c7d-af23-a23c1fdd736b admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:20:04.569734 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e6063-7dd4-4c7d-af23-a23c1fdd736b admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:20:04.575019 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027dcea9-8abf-4cb2-9662-0525a31a2f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:20:04.575827 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027dcea9-8abf-4cb2-9662-0525a31a2f0f admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:04.576119 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027dcea9-8abf-4cb2-9662-0525a31a2f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:04.576420 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027dcea9-8abf-4cb2-9662-0525a31a2f0f admin admin] PATH_INFO: `/v3/projects/405cc415cef743ab94c050cf6a4ddc33/users/80f37d962eae49d2b899144709388cc7/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:04.578949 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:20:04.596195 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027dcea9-8abf-4cb2-9662-0525a31a2f0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=405cc415cef743ab94c050cf6a4ddc33, user_id=80f37d962eae49d2b899144709388cc7, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:20:04.598836 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027dcea9-8abf-4cb2-9662-0525a31a2f0f admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:20:04.612545 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1451/2916] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 15:20:04 2022] PUT /identity/v3/projects/405cc415cef743ab94c050cf6a4ddc33/users/80f37d962eae49d2b899144709388cc7/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:20:04.624300 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4455fda5-27ce-4623-9af9-a80a78093f7f None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:04.624586 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4455fda5-27ce-4623-9af9-a80a78093f7f None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:04.624861 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4455fda5-27ce-4623-9af9-a80a78093f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:04.652631 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-4455fda5-27ce-4623-9af9-a80a78093f7f None None] MFA Rules not processed for user `80f37d962eae49d2b899144709388cc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:20:04.653845 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-4455fda5-27ce-4623-9af9-a80a78093f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:20:04.719674 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1466/2917] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:20:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:20:04.730938 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027dcea9-8abf-4cb2-9662-0525a31a2f0f admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:20:04.732683 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-027dcea9-8abf-4cb2-9662-0525a31a2f0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:20:04.787722 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027dcea9-8abf-4cb2-9662-0525a31a2f0f admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:20:04.792009 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc2bea5-bf29-439e-8b80-f5cb0e4856c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:20:04.792603 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc2bea5-bf29-439e-8b80-f5cb0e4856c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:04.792885 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc2bea5-bf29-439e-8b80-f5cb0e4856c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:04.793160 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc2bea5-bf29-439e-8b80-f5cb0e4856c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:04.801350 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc2bea5-bf29-439e-8b80-f5cb0e4856c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:20:04.802777 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc2bea5-bf29-439e-8b80-f5cb0e4856c7 service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:20:04.812113 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1452/2918] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:20:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:20:14.716608 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=81,keystone:INSERT=3 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:20:14.779752 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4455fda5-27ce-4623-9af9-a80a78093f7f None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:20:14.808081 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=33,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:20:14.843118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4455fda5-27ce-4623-9af9-a80a78093f7f None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:20:14.847679 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adcffb3-d3fe-4eeb-9255-79e372823490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:20:14.848304 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcffb3-d3fe-4eeb-9255-79e372823490 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:14.848586 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcffb3-d3fe-4eeb-9255-79e372823490 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:14.848860 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcffb3-d3fe-4eeb-9255-79e372823490 admin admin] PATH_INFO: `/v3/users/80f37d962eae49d2b899144709388cc7` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:14.849990 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adcffb3-d3fe-4eeb-9255-79e372823490 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80f37d962eae49d2b899144709388cc7)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:20:14.851944 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adcffb3-d3fe-4eeb-9255-79e372823490 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:20:14.898149 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-7adcffb3-d3fe-4eeb-9255-79e372823490 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80f37d962eae49d2b899144709388cc7'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:20:14.898799 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-7adcffb3-d3fe-4eeb-9255-79e372823490 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80f37d962eae49d2b899144709388cc7'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:20:14.906029 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-7adcffb3-d3fe-4eeb-9255-79e372823490 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80f37d962eae49d2b899144709388cc7'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:20:14.912978 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-7adcffb3-d3fe-4eeb-9255-79e372823490 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80f37d962eae49d2b899144709388cc7'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:20:14.913628 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-7adcffb3-d3fe-4eeb-9255-79e372823490 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80f37d962eae49d2b899144709388cc7'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:20:14.932203 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1467/2919] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 15:20:14 2022] DELETE /identity/v3/users/80f37d962eae49d2b899144709388cc7 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:20:14.936671 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbbf297-37bd-4d1c-a98b-0a8a9d22edd5 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:14.936956 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbbf297-37bd-4d1c-a98b-0a8a9d22edd5 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:14.937226 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbbf297-37bd-4d1c-a98b-0a8a9d22edd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:14.956566 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-8dbbf297-37bd-4d1c-a98b-0a8a9d22edd5 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:20:14.957758 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-8dbbf297-37bd-4d1c-a98b-0a8a9d22edd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:20:15.019155 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1453/2920] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:20:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:20:15.028798 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adcffb3-d3fe-4eeb-9255-79e372823490 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:20:15.030212 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-7adcffb3-d3fe-4eeb-9255-79e372823490 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:20:15.088578 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adcffb3-d3fe-4eeb-9255-79e372823490 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:20:15.092229 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18c7e00-28d9-4d02-a3a5-020de3ee1d08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:20:15.092997 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c7e00-28d9-4d02-a3a5-020de3ee1d08 service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:15.093279 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c7e00-28d9-4d02-a3a5-020de3ee1d08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:15.093548 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c7e00-28d9-4d02-a3a5-020de3ee1d08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:15.102527 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18c7e00-28d9-4d02-a3a5-020de3ee1d08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:20:15.104136 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18c7e00-28d9-4d02-a3a5-020de3ee1d08 service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:20:15.113823 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1468/2921] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:20:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:20:15.377719 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbbf297-37bd-4d1c-a98b-0a8a9d22edd5 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:20:15.379186 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-8dbbf297-37bd-4d1c-a98b-0a8a9d22edd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:20:15.391795 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbbf297-37bd-4d1c-a98b-0a8a9d22edd5 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:20:15.396449 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f64a128-103d-4832-a078-521da65b60ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:20:15.397081 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64a128-103d-4832-a078-521da65b60ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:15.397364 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64a128-103d-4832-a078-521da65b60ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:15.397636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64a128-103d-4832-a078-521da65b60ba admin admin] PATH_INFO: `/v3/projects/405cc415cef743ab94c050cf6a4ddc33` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:15.398778 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f64a128-103d-4832-a078-521da65b60ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=405cc415cef743ab94c050cf6a4ddc33)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:20:15.400849 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f64a128-103d-4832-a078-521da65b60ba admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:20:15.448557 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-6f64a128-103d-4832-a078-521da65b60ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '405cc415cef743ab94c050cf6a4ddc33', 'request_id': 'req-6f64a128-103d-4832-a078-521da65b60ba'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:20:15.454615 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-6f64a128-103d-4832-a078-521da65b60ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '405cc415cef743ab94c050cf6a4ddc33', 'request_id': 'req-6f64a128-103d-4832-a078-521da65b60ba'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:20:15.460750 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-6f64a128-103d-4832-a078-521da65b60ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '405cc415cef743ab94c050cf6a4ddc33', 'request_id': 'req-6f64a128-103d-4832-a078-521da65b60ba'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:20:15.472930 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-6f64a128-103d-4832-a078-521da65b60ba admin admin] The token cache is being invalidate because project 405cc415cef743ab94c050cf6a4ddc33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:20:15.473225 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-6f64a128-103d-4832-a078-521da65b60ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:20:15.473842 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-6f64a128-103d-4832-a078-521da65b60ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:20:15.475406 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1454/2922] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 15:20:15 2022] DELETE /identity/v3/projects/405cc415cef743ab94c050cf6a4ddc33 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:20:15.481152 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca5654-bba0-4794-adc3-c73168c652eb None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:15.481435 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca5654-bba0-4794-adc3-c73168c652eb None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:15.481706 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca5654-bba0-4794-adc3-c73168c652eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:15.501349 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-8bca5654-bba0-4794-adc3-c73168c652eb None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:20:15.502550 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-8bca5654-bba0-4794-adc3-c73168c652eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:20:15.519002 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:20:15.563255 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1469/2923] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:20:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:20:15.571120 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef496085-e2a7-4a2c-8335-db4a3416f039 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:15.571453 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef496085-e2a7-4a2c-8335-db4a3416f039 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:15.571806 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef496085-e2a7-4a2c-8335-db4a3416f039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:15.592690 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-ef496085-e2a7-4a2c-8335-db4a3416f039 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:20:15.593893 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-ef496085-e2a7-4a2c-8335-db4a3416f039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:20:15.601964 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1455/2924] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:20:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:20:15.606916 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca5654-bba0-4794-adc3-c73168c652eb None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:20:15.616549 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca5654-bba0-4794-adc3-c73168c652eb None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:20:15.621522 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6b0efe-2c72-4452-a297-5f856340f058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:20:15.622134 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b0efe-2c72-4452-a297-5f856340f058 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:15.622444 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b0efe-2c72-4452-a297-5f856340f058 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:15.622721 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b0efe-2c72-4452-a297-5f856340f058 admin admin] PATH_INFO: `/v3/domains` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:15.623411 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6b0efe-2c72-4452-a297-5f856340f058 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81690) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 15:20:15.623715 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6b0efe-2c72-4452-a297-5f856340f058 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:20:15.625410 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6b0efe-2c72-4452-a297-5f856340f058 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:20:15.637305 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1470/2925] 10.222.0.30 () {58 vars in 1153 bytes} [Tue Aug 30 15:20:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 15:20:15.641709 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef496085-e2a7-4a2c-8335-db4a3416f039 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:20:15.652096 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef496085-e2a7-4a2c-8335-db4a3416f039 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:20:15.656761 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d009e-e194-4cc8-bc3a-89290cc17e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:20:15.657351 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d009e-e194-4cc8-bc3a-89290cc17e49 admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:15.657624 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d009e-e194-4cc8-bc3a-89290cc17e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:15.657890 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d009e-e194-4cc8-bc3a-89290cc17e49 admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:15.658510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5d009e-e194-4cc8-bc3a-89290cc17e49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:20:15.660410 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5d009e-e194-4cc8-bc3a-89290cc17e49 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:20:15.688144 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1456/2926] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 15:20:15 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:20:15.692409 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6b0efe-2c72-4452-a297-5f856340f058 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:20:15.702649 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6b0efe-2c72-4452-a297-5f856340f058 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:20:15.758147 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c682f8-fd2c-4746-adeb-9d02c3df1f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:20:15.758781 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c682f8-fd2c-4746-adeb-9d02c3df1f7a admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:15.759001 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c682f8-fd2c-4746-adeb-9d02c3df1f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:15.759301 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c682f8-fd2c-4746-adeb-9d02c3df1f7a admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:15.760114 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c682f8-fd2c-4746-adeb-9d02c3df1f7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:20:15.762011 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c682f8-fd2c-4746-adeb-9d02c3df1f7a admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:20:15.781868 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1471/2927] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 15:20:15 2022] POST /identity/v3/users => generated 289 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:20:15.786128 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d009e-e194-4cc8-bc3a-89290cc17e49 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:20:15.797050 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d009e-e194-4cc8-bc3a-89290cc17e49 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:20:15.801619 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b24aa7-7557-403d-89c6-35f9eed5fae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:20:15.802210 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b24aa7-7557-403d-89c6-35f9eed5fae4 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:15.802489 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b24aa7-7557-403d-89c6-35f9eed5fae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:15.802761 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b24aa7-7557-403d-89c6-35f9eed5fae4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:15.803504 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b24aa7-7557-403d-89c6-35f9eed5fae4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:20:15.805127 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b24aa7-7557-403d-89c6-35f9eed5fae4 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:20:15.815040 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1457/2928] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:20:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:20:15.818788 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c682f8-fd2c-4746-adeb-9d02c3df1f7a admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:20:15.829036 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c682f8-fd2c-4746-adeb-9d02c3df1f7a admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:20:15.833557 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821de4a3-e7ed-4562-9ca0-a1539b8bf848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:20:15.834137 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821de4a3-e7ed-4562-9ca0-a1539b8bf848 admin admin] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:15.834413 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821de4a3-e7ed-4562-9ca0-a1539b8bf848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:15.834681 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821de4a3-e7ed-4562-9ca0-a1539b8bf848 admin admin] PATH_INFO: `/v3/roles` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:15.835357 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821de4a3-e7ed-4562-9ca0-a1539b8bf848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:20:15.836931 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821de4a3-e7ed-4562-9ca0-a1539b8bf848 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:20:15.847970 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1472/2929] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:20:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:20:15.851924 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b24aa7-7557-403d-89c6-35f9eed5fae4 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:20:15.862573 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b24aa7-7557-403d-89c6-35f9eed5fae4 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:20:15.867172 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3196809c-374c-4bd3-b629-6b28052ef247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:20:15.867850 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3196809c-374c-4bd3-b629-6b28052ef247 admin admin] REQUEST_METHOD: `PUT` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:15.868256 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3196809c-374c-4bd3-b629-6b28052ef247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:15.868417 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3196809c-374c-4bd3-b629-6b28052ef247 admin admin] PATH_INFO: `/v3/projects/b45f3223ea9541f38831d425cb35dbb3/users/2dc2d1a5bde44defae89c15d60172d4b/roles/6a985d5659a3416cba33ca9d30b39f85` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:15.883343 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3196809c-374c-4bd3-b629-6b28052ef247 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b45f3223ea9541f38831d425cb35dbb3, user_id=2dc2d1a5bde44defae89c15d60172d4b, role_id=6a985d5659a3416cba33ca9d30b39f85)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:20:15.886507 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3196809c-374c-4bd3-b629-6b28052ef247 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:20:15.898688 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1458/2930] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 15:20:15 2022] PUT /identity/v3/projects/b45f3223ea9541f38831d425cb35dbb3/users/2dc2d1a5bde44defae89c15d60172d4b/roles/6a985d5659a3416cba33ca9d30b39f85 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:20:15.904497 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a0ff19-9bb5-4d14-8d09-283ca864b8e3 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:15.904785 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a0ff19-9bb5-4d14-8d09-283ca864b8e3 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:15.905057 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a0ff19-9bb5-4d14-8d09-283ca864b8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:15.923784 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-87a0ff19-9bb5-4d14-8d09-283ca864b8e3 None None] MFA Rules not processed for user `4a989d2c0a4a49eb9303e4db6cc0b7e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:20:15.925139 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-87a0ff19-9bb5-4d14-8d09-283ca864b8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:20:15.984316 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1473/2931] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:20:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:20:15.993764 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3196809c-374c-4bd3-b629-6b28052ef247 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:20:15.995623 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-3196809c-374c-4bd3-b629-6b28052ef247 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:20:16.017430 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:20:16.053570 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3196809c-374c-4bd3-b629-6b28052ef247 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:20:16.057211 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f639b7be-323d-4a5c-99b6-97af5841c872 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:20:16.057799 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f639b7be-323d-4a5c-99b6-97af5841c872 service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:16.058071 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f639b7be-323d-4a5c-99b6-97af5841c872 service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:16.058336 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f639b7be-323d-4a5c-99b6-97af5841c872 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:16.067057 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f639b7be-323d-4a5c-99b6-97af5841c872 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:20:16.068451 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f639b7be-323d-4a5c-99b6-97af5841c872 service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:20:16.078487 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1459/2932] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:20:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:20:20.032725 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996401e-64a9-4852-88f7-42706f89cb0e None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:20.033568 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996401e-64a9-4852-88f7-42706f89cb0e None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:20.034014 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996401e-64a9-4852-88f7-42706f89cb0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:20.080271 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-6996401e-64a9-4852-88f7-42706f89cb0e None None] MFA Rules not processed for user `2dc2d1a5bde44defae89c15d60172d4b`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:20:20.081852 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-6996401e-64a9-4852-88f7-42706f89cb0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:20:20.101978 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:20:20.165569 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1474/2933] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:20:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:20:20.170946 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f639b7be-323d-4a5c-99b6-97af5841c872 service neutron] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:20:20.181326 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f639b7be-323d-4a5c-99b6-97af5841c872 service neutron] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:20:20.192787 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18064176-6658-4686-b6cd-e35b000cb71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:20:20.193437 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18064176-6658-4686-b6cd-e35b000cb71e admin admin] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:20.193699 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18064176-6658-4686-b6cd-e35b000cb71e admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:20.194135 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18064176-6658-4686-b6cd-e35b000cb71e admin admin] PATH_INFO: `/v3/projects` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:20.194932 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18064176-6658-4686-b6cd-e35b000cb71e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:20:20.197702 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18064176-6658-4686-b6cd-e35b000cb71e admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:20:20.238267 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1460/2934] 10.222.0.30 () {60 vars in 1153 bytes} [Tue Aug 30 15:20:20 2022] POST /identity/v3/projects => generated 377 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 15:20:20.239577 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6996401e-64a9-4852-88f7-42706f89cb0e None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:20:20.252719 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6996401e-64a9-4852-88f7-42706f89cb0e None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:20:20.314333 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33910826-25de-4927-a514-87ffec1aca9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:20:20.314944 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33910826-25de-4927-a514-87ffec1aca9a admin admin] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:20.315247 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33910826-25de-4927-a514-87ffec1aca9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:20.315540 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33910826-25de-4927-a514-87ffec1aca9a admin admin] PATH_INFO: `/v3/users` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:20.316174 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33910826-25de-4927-a514-87ffec1aca9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:20:20.318084 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33910826-25de-4927-a514-87ffec1aca9a admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:20:20.345526 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1475/2935] 10.222.0.30 () {60 vars in 1144 bytes} [Tue Aug 30 15:20:20 2022] POST /identity/v3/users => generated 295 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 15:20:20.346501 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18064176-6658-4686-b6cd-e35b000cb71e admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:20:20.358325 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18064176-6658-4686-b6cd-e35b000cb71e admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:20:20.363371 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07b29f4-c106-43e2-8b52-10d16e7330c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:20:20.364027 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b29f4-c106-43e2-8b52-10d16e7330c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:20.364305 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b29f4-c106-43e2-8b52-10d16e7330c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:20.364579 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b29f4-c106-43e2-8b52-10d16e7330c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:20.365228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07b29f4-c106-43e2-8b52-10d16e7330c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:20:20.366821 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07b29f4-c106-43e2-8b52-10d16e7330c5 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:20:20.378605 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1461/2936] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:20:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:20:20.383160 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33910826-25de-4927-a514-87ffec1aca9a admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:20:20.394746 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33910826-25de-4927-a514-87ffec1aca9a admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:20:20.400077 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee493723-3532-442f-9c96-8f85101c94ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:20:20.401519 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee493723-3532-442f-9c96-8f85101c94ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:20.401769 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee493723-3532-442f-9c96-8f85101c94ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:20.402118 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee493723-3532-442f-9c96-8f85101c94ed admin admin] PATH_INFO: `/v3/projects/dfc35a56b0ed4a6c9015e2804059eeb1/users/c8f6b756b74242d092c02933c85d7488/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:20.418793 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee493723-3532-442f-9c96-8f85101c94ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfc35a56b0ed4a6c9015e2804059eeb1, user_id=c8f6b756b74242d092c02933c85d7488, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:20:20.421524 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee493723-3532-442f-9c96-8f85101c94ed admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:20:20.441686 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1476/2937] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 15:20:20 2022] PUT /identity/v3/projects/dfc35a56b0ed4a6c9015e2804059eeb1/users/c8f6b756b74242d092c02933c85d7488/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:20:20.443063 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07b29f4-c106-43e2-8b52-10d16e7330c5 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:20:20.457761 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07b29f4-c106-43e2-8b52-10d16e7330c5 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:20:20.518826 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b1ed0e-9c15-48b2-9dbe-06889e572911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:20:20.519731 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1ed0e-9c15-48b2-9dbe-06889e572911 admin admin] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:20.520011 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1ed0e-9c15-48b2-9dbe-06889e572911 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:20.520280 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1ed0e-9c15-48b2-9dbe-06889e572911 admin admin] PATH_INFO: `/v3/roles` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:20.520925 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b1ed0e-9c15-48b2-9dbe-06889e572911 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:20:20.522547 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b1ed0e-9c15-48b2-9dbe-06889e572911 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:20:20.533994 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1462/2938] 10.222.0.30 () {58 vars in 1122 bytes} [Tue Aug 30 15:20:20 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 15:20:20.539609 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee493723-3532-442f-9c96-8f85101c94ed admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:20:20.550550 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee493723-3532-442f-9c96-8f85101c94ed admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:20:20.555478 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e3548-cb4e-4a4c-a2d7-95b24f3c9b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:20:20.556154 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e3548-cb4e-4a4c-a2d7-95b24f3c9b6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:20.556512 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e3548-cb4e-4a4c-a2d7-95b24f3c9b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:20.556951 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e3548-cb4e-4a4c-a2d7-95b24f3c9b6e admin admin] PATH_INFO: `/v3/projects/dfc35a56b0ed4a6c9015e2804059eeb1/users/c8f6b756b74242d092c02933c85d7488/roles/3cda0615d87d4af0860a2a5e9b4e2c05` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:20.558973 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254e3548-cb4e-4a4c-a2d7-95b24f3c9b6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfc35a56b0ed4a6c9015e2804059eeb1, user_id=c8f6b756b74242d092c02933c85d7488, role_id=3cda0615d87d4af0860a2a5e9b4e2c05)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:20:20.561640 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254e3548-cb4e-4a4c-a2d7-95b24f3c9b6e admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:20:20.572520 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1477/2939] 10.222.0.30 () {60 vars in 1483 bytes} [Tue Aug 30 15:20:20 2022] PUT /identity/v3/projects/dfc35a56b0ed4a6c9015e2804059eeb1/users/c8f6b756b74242d092c02933c85d7488/roles/3cda0615d87d4af0860a2a5e9b4e2c05 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:20:25.052775 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4978d1d-3503-4a6c-996c-bd85f976ac2d None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:25.053261 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4978d1d-3503-4a6c-996c-bd85f976ac2d None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:25.053395 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4978d1d-3503-4a6c-996c-bd85f976ac2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:25.083144 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-f4978d1d-3503-4a6c-996c-bd85f976ac2d None None] MFA Rules not processed for user `c8f6b756b74242d092c02933c85d7488`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:20:25.084407 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-f4978d1d-3503-4a6c-996c-bd85f976ac2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:20:25.156487 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1463/2940] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:20:25 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:20:25.172364 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e3548-cb4e-4a4c-a2d7-95b24f3c9b6e admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:20:25.174202 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-254e3548-cb4e-4a4c-a2d7-95b24f3c9b6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:20:25.254000 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e3548-cb4e-4a4c-a2d7-95b24f3c9b6e admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:20:25.259724 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a586221-dacd-4f4f-877c-2ca66da21f67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:20:25.260493 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a586221-dacd-4f4f-877c-2ca66da21f67 service nova] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:25.260773 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a586221-dacd-4f4f-877c-2ca66da21f67 service nova] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:25.261047 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a586221-dacd-4f4f-877c-2ca66da21f67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:25.266165 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:20:25.318813 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a586221-dacd-4f4f-877c-2ca66da21f67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:20:25.320434 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a586221-dacd-4f4f-877c-2ca66da21f67 service nova] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:20:25.331005 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1478/2941] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:20:25 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:20:35.153002 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=94,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:20:35.293653 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e212b31-036c-4778-991e-7c0ffb38506a None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:35.293942 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e212b31-036c-4778-991e-7c0ffb38506a None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:35.294212 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e212b31-036c-4778-991e-7c0ffb38506a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:35.314168 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-0e212b31-036c-4778-991e-7c0ffb38506a None None] MFA Rules not processed for user `f60bee471e954932a928ab2cf7d66c82`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:20:35.315416 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-0e212b31-036c-4778-991e-7c0ffb38506a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:20:35.325804 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=24 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:20:35.370570 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1464/2942] 10.222.0.30 () {58 vars in 967 bytes} [Tue Aug 30 15:20:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 15:20:35.417313 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a586221-dacd-4f4f-877c-2ca66da21f67 service nova] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:20:35.418780 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-6a586221-dacd-4f4f-877c-2ca66da21f67 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:20:35.490980 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a586221-dacd-4f4f-877c-2ca66da21f67 service nova] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:20:35.496088 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb2ed10-4402-4803-9c35-2bbb3acc6fd7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a18b13e3708144e1b5e8df80ec3a4e83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:20:35.496723 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb2ed10-4402-4803-9c35-2bbb3acc6fd7 service swift] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:35.497006 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb2ed10-4402-4803-9c35-2bbb3acc6fd7 service swift] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:35.497279 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb2ed10-4402-4803-9c35-2bbb3acc6fd7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:35.505284 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb2ed10-4402-4803-9c35-2bbb3acc6fd7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:20:35.506859 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb2ed10-4402-4803-9c35-2bbb3acc6fd7 service swift] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:20:35.515541 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1479/2943] 10.222.0.30 () {60 vars in 1384 bytes} [Tue Aug 30 15:20:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 15:20:41.834868 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e212b31-036c-4778-991e-7c0ffb38506a None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:20:41.836306 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-0e212b31-036c-4778-991e-7c0ffb38506a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:20:41.899995 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb2ed10-4402-4803-9c35-2bbb3acc6fd7 service swift] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:20:41.902187 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-1eb2ed10-4402-4803-9c35-2bbb3acc6fd7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:20:41.925786 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e212b31-036c-4778-991e-7c0ffb38506a None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:20:41.932487 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a38bb0d-18d9-400f-9ed7-0bd3d9d7db6a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcf302a170934c25b39699088dac326a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:20:41.933248 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a38bb0d-18d9-400f-9ed7-0bd3d9d7db6a service placement] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:41.933824 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a38bb0d-18d9-400f-9ed7-0bd3d9d7db6a service placement] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:41.934107 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a38bb0d-18d9-400f-9ed7-0bd3d9d7db6a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:41.936314 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-3a38bb0d-18d9-400f-9ed7-0bd3d9d7db6a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:20:41.951325 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a38bb0d-18d9-400f-9ed7-0bd3d9d7db6a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:20:41.952971 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a38bb0d-18d9-400f-9ed7-0bd3d9d7db6a service placement] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:20:41.965126 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1465/2944] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:20:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:20:41.978206 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb2ed10-4402-4803-9c35-2bbb3acc6fd7 service swift] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:20:41.986695 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f3fc45-5e97-4456-8d8d-e51391729577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:20:41.987055 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3fc45-5e97-4456-8d8d-e51391729577 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:41.987379 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3fc45-5e97-4456-8d8d-e51391729577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:41.987658 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3fc45-5e97-4456-8d8d-e51391729577 admin admin] PATH_INFO: `/v3/users/93a00d9af55a42fa8d5628fa81dcae80` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:41.988870 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f3fc45-5e97-4456-8d8d-e51391729577 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93a00d9af55a42fa8d5628fa81dcae80)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:20:41.990858 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f3fc45-5e97-4456-8d8d-e51391729577 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:20:42.056206 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-68f3fc45-5e97-4456-8d8d-e51391729577 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93a00d9af55a42fa8d5628fa81dcae80'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:20:42.057036 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-68f3fc45-5e97-4456-8d8d-e51391729577 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93a00d9af55a42fa8d5628fa81dcae80'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:20:42.064346 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-68f3fc45-5e97-4456-8d8d-e51391729577 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93a00d9af55a42fa8d5628fa81dcae80'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:20:42.071701 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-68f3fc45-5e97-4456-8d8d-e51391729577 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93a00d9af55a42fa8d5628fa81dcae80'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:20:42.072345 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-68f3fc45-5e97-4456-8d8d-e51391729577 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93a00d9af55a42fa8d5628fa81dcae80'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:20:42.091734 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1480/2945] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 15:20:41 2022] DELETE /identity/v3/users/93a00d9af55a42fa8d5628fa81dcae80 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:20:42.519577 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a38bb0d-18d9-400f-9ed7-0bd3d9d7db6a service placement] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:20:42.530014 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a38bb0d-18d9-400f-9ed7-0bd3d9d7db6a service placement] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:20:42.588636 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f5a167-9905-4a6b-9d99-80e75ac913a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:20:42.589258 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f5a167-9905-4a6b-9d99-80e75ac913a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:42.589533 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f5a167-9905-4a6b-9d99-80e75ac913a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:42.589799 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f5a167-9905-4a6b-9d99-80e75ac913a6 admin admin] PATH_INFO: `/v3/projects/4edd140cb317483e8e7e3f04cdee048c` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:42.590909 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f5a167-9905-4a6b-9d99-80e75ac913a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4edd140cb317483e8e7e3f04cdee048c)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:20:42.592845 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f5a167-9905-4a6b-9d99-80e75ac913a6 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:20:42.629311 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:20:42.639073 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b9f5a167-9905-4a6b-9d99-80e75ac913a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4edd140cb317483e8e7e3f04cdee048c', 'request_id': 'req-b9f5a167-9905-4a6b-9d99-80e75ac913a6'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:20:42.644424 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b9f5a167-9905-4a6b-9d99-80e75ac913a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4edd140cb317483e8e7e3f04cdee048c', 'request_id': 'req-b9f5a167-9905-4a6b-9d99-80e75ac913a6'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:20:42.650582 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b9f5a167-9905-4a6b-9d99-80e75ac913a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4edd140cb317483e8e7e3f04cdee048c', 'request_id': 'req-b9f5a167-9905-4a6b-9d99-80e75ac913a6'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:20:42.662227 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b9f5a167-9905-4a6b-9d99-80e75ac913a6 admin admin] The token cache is being invalidate because project 4edd140cb317483e8e7e3f04cdee048c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81689) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:20:42.662520 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b9f5a167-9905-4a6b-9d99-80e75ac913a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:20:42.663134 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-b9f5a167-9905-4a6b-9d99-80e75ac913a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:20:42.664633 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1466/2946] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 15:20:42 2022] DELETE /identity/v3/projects/4edd140cb317483e8e7e3f04cdee048c => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:20:42.668712 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f3fc45-5e97-4456-8d8d-e51391729577 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:20:42.670187 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-68f3fc45-5e97-4456-8d8d-e51391729577 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:20:42.684021 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f3fc45-5e97-4456-8d8d-e51391729577 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:20:42.688563 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:20:42.741763 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e965d7cc-0491-4970-9650-2884e57cbe33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:20:42.742386 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e965d7cc-0491-4970-9650-2884e57cbe33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:42.742665 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e965d7cc-0491-4970-9650-2884e57cbe33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:42.742938 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e965d7cc-0491-4970-9650-2884e57cbe33 admin admin] PATH_INFO: `/v3/users/519d88ca754b4a23a9b7958a87c40150` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:42.744686 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e965d7cc-0491-4970-9650-2884e57cbe33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=519d88ca754b4a23a9b7958a87c40150)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:20:42.746587 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e965d7cc-0491-4970-9650-2884e57cbe33 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:20:42.792194 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-e965d7cc-0491-4970-9650-2884e57cbe33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '519d88ca754b4a23a9b7958a87c40150'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:20:42.792835 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-e965d7cc-0491-4970-9650-2884e57cbe33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '519d88ca754b4a23a9b7958a87c40150'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:20:42.800410 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-e965d7cc-0491-4970-9650-2884e57cbe33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '519d88ca754b4a23a9b7958a87c40150'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:20:42.808490 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-e965d7cc-0491-4970-9650-2884e57cbe33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '519d88ca754b4a23a9b7958a87c40150'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:20:42.809158 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-e965d7cc-0491-4970-9650-2884e57cbe33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '519d88ca754b4a23a9b7958a87c40150'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:20:42.827853 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1481/2947] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 15:20:42 2022] DELETE /identity/v3/users/519d88ca754b4a23a9b7958a87c40150 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:20:42.937050 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15a260-1247-4de3-8348-e0cdb727f458 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:42.937564 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15a260-1247-4de3-8348-e0cdb727f458 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:42.937643 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15a260-1247-4de3-8348-e0cdb727f458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:42.956657 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-bf15a260-1247-4de3-8348-e0cdb727f458 None None] MFA Rules not processed for user `2dc2d1a5bde44defae89c15d60172d4b`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:20:42.957869 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-bf15a260-1247-4de3-8348-e0cdb727f458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:20:43.001300 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e965d7cc-0491-4970-9650-2884e57cbe33 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:20:43.002836 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-e965d7cc-0491-4970-9650-2884e57cbe33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:20:43.033492 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1467/2948] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:20:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:20:43.082590 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e965d7cc-0491-4970-9650-2884e57cbe33 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:20:43.087688 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5268b1d0-b9b7-4be7-b601-8ce1dc3d2b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:20:43.088318 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5268b1d0-b9b7-4be7-b601-8ce1dc3d2b00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:43.088595 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5268b1d0-b9b7-4be7-b601-8ce1dc3d2b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:43.088870 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5268b1d0-b9b7-4be7-b601-8ce1dc3d2b00 admin admin] PATH_INFO: `/v3/projects/b2a376925fda431ea5ce67cc7896d1a6` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:43.089997 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5268b1d0-b9b7-4be7-b601-8ce1dc3d2b00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2a376925fda431ea5ce67cc7896d1a6)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:20:43.091980 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5268b1d0-b9b7-4be7-b601-8ce1dc3d2b00 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:20:43.128024 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:20:43.142986 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-5268b1d0-b9b7-4be7-b601-8ce1dc3d2b00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2a376925fda431ea5ce67cc7896d1a6', 'request_id': 'req-5268b1d0-b9b7-4be7-b601-8ce1dc3d2b00'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:20:43.148526 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-5268b1d0-b9b7-4be7-b601-8ce1dc3d2b00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2a376925fda431ea5ce67cc7896d1a6', 'request_id': 'req-5268b1d0-b9b7-4be7-b601-8ce1dc3d2b00'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:20:43.154976 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-5268b1d0-b9b7-4be7-b601-8ce1dc3d2b00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2a376925fda431ea5ce67cc7896d1a6', 'request_id': 'req-5268b1d0-b9b7-4be7-b601-8ce1dc3d2b00'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:20:43.168627 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-5268b1d0-b9b7-4be7-b601-8ce1dc3d2b00 admin admin] The token cache is being invalidate because project b2a376925fda431ea5ce67cc7896d1a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:20:43.168627 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-5268b1d0-b9b7-4be7-b601-8ce1dc3d2b00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:20:43.169221 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-5268b1d0-b9b7-4be7-b601-8ce1dc3d2b00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:20:43.170710 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1482/2949] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 15:20:43 2022] DELETE /identity/v3/projects/b2a376925fda431ea5ce67cc7896d1a6 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:20:53.026101 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=36,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:20:53.164646 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=13,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:20:56.689924 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15a260-1247-4de3-8348-e0cdb727f458 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:20:56.691561 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-bf15a260-1247-4de3-8348-e0cdb727f458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:20:56.760130 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15a260-1247-4de3-8348-e0cdb727f458 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:20:56.812363 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f2080-b2aa-484f-987b-d9cf31c12cd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:20:56.812998 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f2080-b2aa-484f-987b-d9cf31c12cd7 service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:56.813277 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f2080-b2aa-484f-987b-d9cf31c12cd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:56.813547 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f2080-b2aa-484f-987b-d9cf31c12cd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:56.815406 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-788f2080-b2aa-484f-987b-d9cf31c12cd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:20:56.825974 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788f2080-b2aa-484f-987b-d9cf31c12cd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:20:56.827545 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788f2080-b2aa-484f-987b-d9cf31c12cd7 service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:20:56.836018 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1468/2950] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:20:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:20:59.402903 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5268b1d0-b9b7-4be7-b601-8ce1dc3d2b00 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:20:59.404440 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-5268b1d0-b9b7-4be7-b601-8ce1dc3d2b00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:20:59.416695 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5268b1d0-b9b7-4be7-b601-8ce1dc3d2b00 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:20:59.420278 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1bebd0-c3e0-41a1-a09a-07de68c8e164 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:20:59.420868 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1bebd0-c3e0-41a1-a09a-07de68c8e164 service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:20:59.421153 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1bebd0-c3e0-41a1-a09a-07de68c8e164 service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:20:59.421424 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1bebd0-c3e0-41a1-a09a-07de68c8e164 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:20:59.423198 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-1d1bebd0-c3e0-41a1-a09a-07de68c8e164 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:20:59.485927 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1bebd0-c3e0-41a1-a09a-07de68c8e164 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:20:59.487391 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1bebd0-c3e0-41a1-a09a-07de68c8e164 service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:20:59.497104 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1483/2951] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:20:59 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:21:03.538746 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f2080-b2aa-484f-987b-d9cf31c12cd7 service neutron] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:21:03.540687 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-788f2080-b2aa-484f-987b-d9cf31c12cd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:21:03.635144 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f2080-b2aa-484f-987b-d9cf31c12cd7 service neutron] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:21:03.640358 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed857aba-d3ac-472d-a98f-74731dd33d4d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcf302a170934c25b39699088dac326a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:21:03.640959 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed857aba-d3ac-472d-a98f-74731dd33d4d service placement] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:21:03.641232 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed857aba-d3ac-472d-a98f-74731dd33d4d service placement] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:21:03.641497 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed857aba-d3ac-472d-a98f-74731dd33d4d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:21:03.643353 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-ed857aba-d3ac-472d-a98f-74731dd33d4d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:21:03.655799 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed857aba-d3ac-472d-a98f-74731dd33d4d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:21:03.657362 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed857aba-d3ac-472d-a98f-74731dd33d4d service placement] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:21:03.668005 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1469/2952] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:21:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:21:06.621630 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3579511-ed9d-4a0c-8fc3-acd4ee5d7788 None None] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:21:06.622157 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3579511-ed9d-4a0c-8fc3-acd4ee5d7788 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:21:06.622157 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3579511-ed9d-4a0c-8fc3-acd4ee5d7788 None None] PATH_INFO: `/` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:21:06.623755 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1484/2953] 10.222.0.30 () {54 vars in 921 bytes} [Tue Aug 30 15:21:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:21:06.630162 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a839587a-d2f2-48b5-a706-b28e44074d19 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:21:06.630646 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a839587a-d2f2-48b5-a706-b28e44074d19 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:21:06.630762 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a839587a-d2f2-48b5-a706-b28e44074d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:21:06.666918 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=100 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:21:06.670177 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-a839587a-d2f2-48b5-a706-b28e44074d19 None None] MFA Rules not processed for user `9e14e86e085d4987aece69fe64867ce6`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:21:06.671852 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-a839587a-d2f2-48b5-a706-b28e44074d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:21:06.678586 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1470/2954] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Aug 30 15:21:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:21:06.691387 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3579511-ed9d-4a0c-8fc3-acd4ee5d7788 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:21:06.703198 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3579511-ed9d-4a0c-8fc3-acd4ee5d7788 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:21:06.706801 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddee2ad-11e9-4e4e-9464-d66b78b4754b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:21:06.707433 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddee2ad-11e9-4e4e-9464-d66b78b4754b service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:21:06.707713 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddee2ad-11e9-4e4e-9464-d66b78b4754b service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:21:06.707988 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddee2ad-11e9-4e4e-9464-d66b78b4754b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:21:06.715551 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddee2ad-11e9-4e4e-9464-d66b78b4754b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:21:06.716913 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddee2ad-11e9-4e4e-9464-d66b78b4754b service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:21:06.727632 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1485/2955] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:21:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:21:06.809766 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4d0d4-f197-4769-81df-b3595bb2761a None None] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:21:06.810051 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4d0d4-f197-4769-81df-b3595bb2761a None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:21:06.810328 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4d0d4-f197-4769-81df-b3595bb2761a None None] PATH_INFO: `/` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:21:06.811732 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1471/2956] 10.222.0.30 () {54 vars in 921 bytes} [Tue Aug 30 15:21:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 15:21:06.816615 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb8f2d-1e1f-447a-9fab-d1864c3b7893 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:21:06.816896 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb8f2d-1e1f-447a-9fab-d1864c3b7893 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:21:06.817190 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb8f2d-1e1f-447a-9fab-d1864c3b7893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:21:06.855951 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-ceeb8f2d-1e1f-447a-9fab-d1864c3b7893 None None] MFA Rules not processed for user `8878de66a03044e38e507028853a0cd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:21:06.857182 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-ceeb8f2d-1e1f-447a-9fab-d1864c3b7893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:21:06.922302 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1486/2957] 10.222.0.30 () {58 vars in 1018 bytes} [Tue Aug 30 15:21:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 15:21:06.933418 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef4d0d4-f197-4769-81df-b3595bb2761a None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:21:06.944335 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef4d0d4-f197-4769-81df-b3595bb2761a None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:21:06.948167 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f76b4-5b3a-451a-b500-95484ff9e2cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:21:06.948751 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f76b4-5b3a-451a-b500-95484ff9e2cc service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:21:06.949052 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f76b4-5b3a-451a-b500-95484ff9e2cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:21:06.949322 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f76b4-5b3a-451a-b500-95484ff9e2cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:21:06.959759 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448f76b4-5b3a-451a-b500-95484ff9e2cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:21:06.961179 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448f76b4-5b3a-451a-b500-95484ff9e2cc service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:21:06.971664 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1472/2958] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:21:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:21:16.912403 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=73 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:21:16.967055 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=6 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:22:36.419898 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeb8f2d-1e1f-447a-9fab-d1864c3b7893 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:22:36.421466 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-ceeb8f2d-1e1f-447a-9fab-d1864c3b7893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:22:36.434061 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeb8f2d-1e1f-447a-9fab-d1864c3b7893 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:22:36.437675 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ffb3bd-a32c-49aa-8e21-e2d7ddaaf3ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:22:36.438279 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffb3bd-a32c-49aa-8e21-e2d7ddaaf3ee service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:22:36.438561 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffb3bd-a32c-49aa-8e21-e2d7ddaaf3ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:22:36.438834 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffb3bd-a32c-49aa-8e21-e2d7ddaaf3ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:22:36.440720 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-61ffb3bd-a32c-49aa-8e21-e2d7ddaaf3ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:22:36.456477 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ffb3bd-a32c-49aa-8e21-e2d7ddaaf3ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:22:36.457919 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ffb3bd-a32c-49aa-8e21-e2d7ddaaf3ee service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:22:36.470086 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1487/2959] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:22:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:22:45.186826 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f76b4-5b3a-451a-b500-95484ff9e2cc service neutron] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:22:45.188527 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-448f76b4-5b3a-451a-b500-95484ff9e2cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:22:45.286394 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f76b4-5b3a-451a-b500-95484ff9e2cc service neutron] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:22:45.291646 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5875419-b7df-4882-8b50-e6a9e16d9004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:22:45.292511 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5875419-b7df-4882-8b50-e6a9e16d9004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:22:45.292795 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5875419-b7df-4882-8b50-e6a9e16d9004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:22:45.293064 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5875419-b7df-4882-8b50-e6a9e16d9004 admin admin] PATH_INFO: `/v3/users/b2905d19c05b4f53ba70e2bb3d165bc1` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:22:45.294196 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5875419-b7df-4882-8b50-e6a9e16d9004 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2905d19c05b4f53ba70e2bb3d165bc1)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:22:45.296157 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5875419-b7df-4882-8b50-e6a9e16d9004 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:22:45.341906 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-a5875419-b7df-4882-8b50-e6a9e16d9004 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2905d19c05b4f53ba70e2bb3d165bc1'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:22:45.342490 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-a5875419-b7df-4882-8b50-e6a9e16d9004 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2905d19c05b4f53ba70e2bb3d165bc1'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:22:45.354940 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-a5875419-b7df-4882-8b50-e6a9e16d9004 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2905d19c05b4f53ba70e2bb3d165bc1'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:22:45.361690 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-a5875419-b7df-4882-8b50-e6a9e16d9004 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2905d19c05b4f53ba70e2bb3d165bc1'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:22:45.368152 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-a5875419-b7df-4882-8b50-e6a9e16d9004 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2905d19c05b4f53ba70e2bb3d165bc1'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:22:45.375038 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1473/2960] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 15:22:45 2022] DELETE /identity/v3/users/b2905d19c05b4f53ba70e2bb3d165bc1 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:22:45.544957 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ffb3bd-a32c-49aa-8e21-e2d7ddaaf3ee service neutron] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:22:45.580727 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ffb3bd-a32c-49aa-8e21-e2d7ddaaf3ee service neutron] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:22:45.640254 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f1b0b-340e-4649-b099-d979752469ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:22:45.640788 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f1b0b-340e-4649-b099-d979752469ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:22:45.641069 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f1b0b-340e-4649-b099-d979752469ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:22:45.641351 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f1b0b-340e-4649-b099-d979752469ef admin admin] PATH_INFO: `/v3/projects/7034e8a9461846a7b08fe7328ecf7a47` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:22:45.642509 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98f1b0b-340e-4649-b099-d979752469ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=7034e8a9461846a7b08fe7328ecf7a47)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:22:45.644444 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98f1b0b-340e-4649-b099-d979752469ef admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:22:45.691477 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-a98f1b0b-340e-4649-b099-d979752469ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7034e8a9461846a7b08fe7328ecf7a47', 'request_id': 'req-a98f1b0b-340e-4649-b099-d979752469ef'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:22:45.696999 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-a98f1b0b-340e-4649-b099-d979752469ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7034e8a9461846a7b08fe7328ecf7a47', 'request_id': 'req-a98f1b0b-340e-4649-b099-d979752469ef'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:22:45.703338 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-a98f1b0b-340e-4649-b099-d979752469ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7034e8a9461846a7b08fe7328ecf7a47', 'request_id': 'req-a98f1b0b-340e-4649-b099-d979752469ef'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:22:45.716334 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-a98f1b0b-340e-4649-b099-d979752469ef admin admin] The token cache is being invalidate because project 7034e8a9461846a7b08fe7328ecf7a47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:22:45.716624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-a98f1b0b-340e-4649-b099-d979752469ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:22:45.717245 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-a98f1b0b-340e-4649-b099-d979752469ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:22:45.718711 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1488/2961] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 15:22:45 2022] DELETE /identity/v3/projects/7034e8a9461846a7b08fe7328ecf7a47 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:22:45.722775 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5875419-b7df-4882-8b50-e6a9e16d9004 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:22:45.724202 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-a5875419-b7df-4882-8b50-e6a9e16d9004 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:22:45.737849 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5875419-b7df-4882-8b50-e6a9e16d9004 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:22:45.782377 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:22:45.796972 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e34cda-90af-4586-a33c-aecb1883df45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:22:45.797598 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e34cda-90af-4586-a33c-aecb1883df45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:22:45.797882 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e34cda-90af-4586-a33c-aecb1883df45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:22:45.798154 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e34cda-90af-4586-a33c-aecb1883df45 admin admin] PATH_INFO: `/v3/users/61fc09826a874f7f9a9d25238e5dceb8` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:22:45.799322 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e34cda-90af-4586-a33c-aecb1883df45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61fc09826a874f7f9a9d25238e5dceb8)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:22:45.801228 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e34cda-90af-4586-a33c-aecb1883df45 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:22:45.846038 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-43e34cda-90af-4586-a33c-aecb1883df45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61fc09826a874f7f9a9d25238e5dceb8'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:22:45.846678 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-43e34cda-90af-4586-a33c-aecb1883df45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61fc09826a874f7f9a9d25238e5dceb8'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:22:45.858271 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-43e34cda-90af-4586-a33c-aecb1883df45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61fc09826a874f7f9a9d25238e5dceb8'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:22:45.864970 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-43e34cda-90af-4586-a33c-aecb1883df45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61fc09826a874f7f9a9d25238e5dceb8'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:22:45.872556 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-43e34cda-90af-4586-a33c-aecb1883df45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61fc09826a874f7f9a9d25238e5dceb8'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:22:45.878642 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1474/2962] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 15:22:45 2022] DELETE /identity/v3/users/61fc09826a874f7f9a9d25238e5dceb8 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:22:46.048464 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f1b0b-340e-4649-b099-d979752469ef admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:22:46.049948 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-a98f1b0b-340e-4649-b099-d979752469ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:22:46.063571 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f1b0b-340e-4649-b099-d979752469ef admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:22:46.116015 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:22:46.128332 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71124216-c2fe-4fc7-8a2b-e06d2225fcb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:22:46.128967 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71124216-c2fe-4fc7-8a2b-e06d2225fcb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:22:46.129260 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71124216-c2fe-4fc7-8a2b-e06d2225fcb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:22:46.129522 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71124216-c2fe-4fc7-8a2b-e06d2225fcb2 admin admin] PATH_INFO: `/v3/projects/be98d82b0c6b40439bee9af8733cc980` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:22:46.130674 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71124216-c2fe-4fc7-8a2b-e06d2225fcb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be98d82b0c6b40439bee9af8733cc980)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:22:46.132684 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71124216-c2fe-4fc7-8a2b-e06d2225fcb2 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:22:46.183351 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-71124216-c2fe-4fc7-8a2b-e06d2225fcb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be98d82b0c6b40439bee9af8733cc980', 'request_id': 'req-71124216-c2fe-4fc7-8a2b-e06d2225fcb2'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:22:46.188925 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-71124216-c2fe-4fc7-8a2b-e06d2225fcb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be98d82b0c6b40439bee9af8733cc980', 'request_id': 'req-71124216-c2fe-4fc7-8a2b-e06d2225fcb2'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:22:46.195954 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-71124216-c2fe-4fc7-8a2b-e06d2225fcb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be98d82b0c6b40439bee9af8733cc980', 'request_id': 'req-71124216-c2fe-4fc7-8a2b-e06d2225fcb2'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:22:46.211238 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-71124216-c2fe-4fc7-8a2b-e06d2225fcb2 admin admin] The token cache is being invalidate because project be98d82b0c6b40439bee9af8733cc980 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:22:46.211564 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-71124216-c2fe-4fc7-8a2b-e06d2225fcb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:22:46.212473 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-71124216-c2fe-4fc7-8a2b-e06d2225fcb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:22:46.214567 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1489/2963] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 15:22:46 2022] DELETE /identity/v3/projects/be98d82b0c6b40439bee9af8733cc980 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:22:55.875165 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=32,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:22:56.212175 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=27,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:22:57.815770 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062d66c-cc9d-4e1f-ab1c-3c35b0018a98 None None] REQUEST_METHOD: `POST` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:22:57.816299 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062d66c-cc9d-4e1f-ab1c-3c35b0018a98 None None] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:22:57.816299 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062d66c-cc9d-4e1f-ab1c-3c35b0018a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:22:57.837649 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.auth.core [None req-e062d66c-cc9d-4e1f-ab1c-3c35b0018a98 None None] MFA Rules not processed for user `2dc2d1a5bde44defae89c15d60172d4b`. Rule list: `[]` (Enabled: `True`). {{(pid=81689) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:22:57.838884 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-e062d66c-cc9d-4e1f-ab1c-3c35b0018a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:22:57.897374 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1475/2964] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:22:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:22:58.209108 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71124216-c2fe-4fc7-8a2b-e06d2225fcb2 admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:22:58.210630 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-71124216-c2fe-4fc7-8a2b-e06d2225fcb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:22:58.273677 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71124216-c2fe-4fc7-8a2b-e06d2225fcb2 admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:22:58.277611 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd702585-4e27-4e19-9a70-0dcd5770bb7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:22:58.278206 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd702585-4e27-4e19-9a70-0dcd5770bb7e service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:22:58.278485 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd702585-4e27-4e19-9a70-0dcd5770bb7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:22:58.278758 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd702585-4e27-4e19-9a70-0dcd5770bb7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:22:58.280490 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-bd702585-4e27-4e19-9a70-0dcd5770bb7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:22:58.289850 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd702585-4e27-4e19-9a70-0dcd5770bb7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:22:58.292057 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd702585-4e27-4e19-9a70-0dcd5770bb7e service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:22:58.300097 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1490/2965] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:22:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:22:58.741255 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e062d66c-cc9d-4e1f-ab1c-3c35b0018a98 None None] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:22:58.742749 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-e062d66c-cc9d-4e1f-ab1c-3c35b0018a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:22:58.815375 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e062d66c-cc9d-4e1f-ab1c-3c35b0018a98 None None] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:22:58.820096 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e796d-472d-432d-b2bf-9c58baca75e9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcf302a170934c25b39699088dac326a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:22:58.820783 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e796d-472d-432d-b2bf-9c58baca75e9 service placement] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:22:58.821124 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e796d-472d-432d-b2bf-9c58baca75e9 service placement] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:22:58.821397 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e796d-472d-432d-b2bf-9c58baca75e9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:22:58.824238 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-569e796d-472d-432d-b2bf-9c58baca75e9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:22:58.836177 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569e796d-472d-432d-b2bf-9c58baca75e9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:22:58.838055 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569e796d-472d-432d-b2bf-9c58baca75e9 service placement] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:22:58.848069 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1476/2966] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:22:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:23:08.296353 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=29 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:23:08.843818 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=57 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:23:11.050051 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd702585-4e27-4e19-9a70-0dcd5770bb7e service neutron] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:23:11.051518 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-bd702585-4e27-4e19-9a70-0dcd5770bb7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:23:11.143771 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd702585-4e27-4e19-9a70-0dcd5770bb7e service neutron] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:23:11.149748 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebc8794-21de-4ee7-8333-33fee23b69f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:23:11.150361 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc8794-21de-4ee7-8333-33fee23b69f9 service nova] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:23:11.150645 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc8794-21de-4ee7-8333-33fee23b69f9 service nova] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:23:11.150924 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc8794-21de-4ee7-8333-33fee23b69f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:23:11.152735 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-6ebc8794-21de-4ee7-8333-33fee23b69f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:23:11.165895 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebc8794-21de-4ee7-8333-33fee23b69f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:23:11.167684 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebc8794-21de-4ee7-8333-33fee23b69f9 service nova] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:23:11.178138 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1491/2967] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:23:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:23:14.914070 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e796d-472d-432d-b2bf-9c58baca75e9 service placement] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:23:14.927366 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e796d-472d-432d-b2bf-9c58baca75e9 service placement] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:23:14.933742 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e177dc-b190-4b63-a281-df57215fbec8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:23:14.934335 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e177dc-b190-4b63-a281-df57215fbec8 service nova] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:23:14.934610 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e177dc-b190-4b63-a281-df57215fbec8 service nova] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:23:14.934873 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e177dc-b190-4b63-a281-df57215fbec8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:23:14.936683 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-d1e177dc-b190-4b63-a281-df57215fbec8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:23:14.950225 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e177dc-b190-4b63-a281-df57215fbec8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:23:14.951828 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e177dc-b190-4b63-a281-df57215fbec8 service nova] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:23:14.963194 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1477/2968] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:23:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:23:21.173729 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=41 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:23:24.957898 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=6 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:24:52.875958 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb693ed-b4a3-4a17-8a4e-2d0ee1c1c3a4 None None] REQUEST_METHOD: `POST` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:24:52.876562 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb693ed-b4a3-4a17-8a4e-2d0ee1c1c3a4 None None] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:24:52.876562 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb693ed-b4a3-4a17-8a4e-2d0ee1c1c3a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:24:52.896458 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.auth.core [None req-5fb693ed-b4a3-4a17-8a4e-2d0ee1c1c3a4 None None] MFA Rules not processed for user `2dc2d1a5bde44defae89c15d60172d4b`. Rule list: `[]` (Enabled: `True`). {{(pid=81690) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 15:24:52.897667 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-5fb693ed-b4a3-4a17-8a4e-2d0ee1c1c3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:24:52.905504 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1492/2969] 10.222.0.30 () {58 vars in 958 bytes} [Tue Aug 30 15:24:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 15:24:52.957255 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e177dc-b190-4b63-a281-df57215fbec8 service nova] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:24:52.960240 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-d1e177dc-b190-4b63-a281-df57215fbec8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:24:53.052082 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e177dc-b190-4b63-a281-df57215fbec8 service nova] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:24:53.055652 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f9f34c-83a2-4673-b123-8c19deff9722 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6fda7075198d48da86aa9c0dab6ff7ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:24:53.056249 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f9f34c-83a2-4673-b123-8c19deff9722 service cinder] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:24:53.056525 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f9f34c-83a2-4673-b123-8c19deff9722 service cinder] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:24:53.056792 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f9f34c-83a2-4673-b123-8c19deff9722 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:24:53.058496 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-94f9f34c-83a2-4673-b123-8c19deff9722 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:24:53.084034 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f9f34c-83a2-4673-b123-8c19deff9722 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:24:53.085462 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f9f34c-83a2-4673-b123-8c19deff9722 service cinder] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:24:53.096881 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1478/2970] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:24:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:24:56.379541 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb693ed-b4a3-4a17-8a4e-2d0ee1c1c3a4 None None] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:24:56.381139 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-5fb693ed-b4a3-4a17-8a4e-2d0ee1c1c3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:24:56.399531 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb693ed-b4a3-4a17-8a4e-2d0ee1c1c3a4 None None] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:24:56.406056 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98d595-fd87-444a-a097-64cf44541aa2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8878de66a03044e38e507028853a0cd1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:24:56.406717 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98d595-fd87-444a-a097-64cf44541aa2 service nova] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:24:56.406998 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98d595-fd87-444a-a097-64cf44541aa2 service nova] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:24:56.407300 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98d595-fd87-444a-a097-64cf44541aa2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:24:56.409037 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-ac98d595-fd87-444a-a097-64cf44541aa2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:24:56.421650 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac98d595-fd87-444a-a097-64cf44541aa2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:24:56.423300 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac98d595-fd87-444a-a097-64cf44541aa2 service nova] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:24:56.434584 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1493/2971] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:24:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 15:25:03.091385 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=40 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:25:06.429757 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=9 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:26:06.214049 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f9f34c-83a2-4673-b123-8c19deff9722 service cinder] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:26:06.215512 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-94f9f34c-83a2-4673-b123-8c19deff9722 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:26:06.250249 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f9f34c-83a2-4673-b123-8c19deff9722 service cinder] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:26:06.253892 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df5221e-2b36-49dc-914f-4b8a55ad7f22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:26:06.254484 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5221e-2b36-49dc-914f-4b8a55ad7f22 service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:26:06.254762 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5221e-2b36-49dc-914f-4b8a55ad7f22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:26:06.255031 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5221e-2b36-49dc-914f-4b8a55ad7f22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:26:06.256767 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-3df5221e-2b36-49dc-914f-4b8a55ad7f22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:26:06.266168 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df5221e-2b36-49dc-914f-4b8a55ad7f22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:26:06.267658 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df5221e-2b36-49dc-914f-4b8a55ad7f22 service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:26:06.275390 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1479/2972] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:26:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:26:08.551709 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98d595-fd87-444a-a097-64cf44541aa2 service nova] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:26:08.553106 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-ac98d595-fd87-444a-a097-64cf44541aa2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:26:08.568422 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98d595-fd87-444a-a097-64cf44541aa2 service nova] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:26:08.573317 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac101fc1-071d-4200-995c-65c7da97fa12 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcf302a170934c25b39699088dac326a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:26:08.573902 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac101fc1-071d-4200-995c-65c7da97fa12 service placement] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:26:08.574176 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac101fc1-071d-4200-995c-65c7da97fa12 service placement] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:26:08.574443 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac101fc1-071d-4200-995c-65c7da97fa12 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:26:08.576350 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-ac101fc1-071d-4200-995c-65c7da97fa12 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:26:08.588403 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac101fc1-071d-4200-995c-65c7da97fa12 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:26:08.589954 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac101fc1-071d-4200-995c-65c7da97fa12 service placement] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:26:08.600241 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1494/2973] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:26:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:26:16.271858 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=15 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:26:18.596004 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=6 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:26:38.096003 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df5221e-2b36-49dc-914f-4b8a55ad7f22 service neutron] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:26:38.105464 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df5221e-2b36-49dc-914f-4b8a55ad7f22 service neutron] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:26:38.109117 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6600f5f-a3c0-4e0a-a957-808f38423dff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:26:38.109698 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6600f5f-a3c0-4e0a-a957-808f38423dff service neutron] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:26:38.109983 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6600f5f-a3c0-4e0a-a957-808f38423dff service neutron] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:26:38.110253 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6600f5f-a3c0-4e0a-a957-808f38423dff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:26:38.112193 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-c6600f5f-a3c0-4e0a-a957-808f38423dff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:26:38.122750 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6600f5f-a3c0-4e0a-a957-808f38423dff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:26:38.124196 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6600f5f-a3c0-4e0a-a957-808f38423dff service neutron] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:26:38.132959 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1480/2974] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:26:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:26:48.129035 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=6 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:26:51.359356 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac101fc1-071d-4200-995c-65c7da97fa12 service placement] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:26:51.369016 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac101fc1-071d-4200-995c-65c7da97fa12 service placement] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:26:51.372607 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a1b18d-d18f-421b-80c9-594111cb9a03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e14e86e085d4987aece69fe64867ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:26:51.373195 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a1b18d-d18f-421b-80c9-594111cb9a03 service neutron] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:26:51.373469 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a1b18d-d18f-421b-80c9-594111cb9a03 service neutron] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:26:51.373736 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a1b18d-d18f-421b-80c9-594111cb9a03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:26:51.375501 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-d9a1b18d-d18f-421b-80c9-594111cb9a03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:26:51.438859 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a1b18d-d18f-421b-80c9-594111cb9a03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:26:51.440258 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a1b18d-d18f-421b-80c9-594111cb9a03 service neutron] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:26:51.450454 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1495/2975] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:26:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:26:54.497115 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6600f5f-a3c0-4e0a-a957-808f38423dff service neutron] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:26:54.524809 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6600f5f-a3c0-4e0a-a957-808f38423dff service neutron] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:26:54.531510 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fffe3fb-118f-4f7a-943c-75c5972c981c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcf302a170934c25b39699088dac326a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:26:54.532241 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fffe3fb-118f-4f7a-943c-75c5972c981c service placement] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:26:54.532383 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fffe3fb-118f-4f7a-943c-75c5972c981c service placement] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:26:54.532649 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fffe3fb-118f-4f7a-943c-75c5972c981c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:26:54.534413 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-0fffe3fb-118f-4f7a-943c-75c5972c981c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:26:54.548035 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fffe3fb-118f-4f7a-943c-75c5972c981c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:26:54.549825 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fffe3fb-118f-4f7a-943c-75c5972c981c service placement] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:26:54.563443 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1481/2976] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:26:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:26:57.741476 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a1b18d-d18f-421b-80c9-594111cb9a03 service neutron] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:26:57.753206 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a1b18d-d18f-421b-80c9-594111cb9a03 service neutron] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:26:57.758610 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354b345f-a1bf-44d1-b5ae-36b3c0854077 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcf302a170934c25b39699088dac326a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:26:57.759367 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b345f-a1bf-44d1-b5ae-36b3c0854077 service placement] REQUEST_METHOD: `GET` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:26:57.759716 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b345f-a1bf-44d1-b5ae-36b3c0854077 service placement] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:26:57.760057 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b345f-a1bf-44d1-b5ae-36b3c0854077 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:26:57.761897 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-354b345f-a1bf-44d1-b5ae-36b3c0854077 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:26:57.773835 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354b345f-a1bf-44d1-b5ae-36b3c0854077 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:26:57.775515 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354b345f-a1bf-44d1-b5ae-36b3c0854077 service placement] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:26:57.787469 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1496/2977] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:26:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:26:57.803039 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fffe3fb-118f-4f7a-943c-75c5972c981c service placement] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:26:57.804663 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-0fffe3fb-118f-4f7a-943c-75c5972c981c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:26:57.818785 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fffe3fb-118f-4f7a-943c-75c5972c981c service placement] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:26:57.823866 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e10d850-349a-43d0-a7bb-2076979f77f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:26:57.824619 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e10d850-349a-43d0-a7bb-2076979f77f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:26:57.824901 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e10d850-349a-43d0-a7bb-2076979f77f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:26:57.825609 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e10d850-349a-43d0-a7bb-2076979f77f1 admin admin] PATH_INFO: `/v3/users/2dc2d1a5bde44defae89c15d60172d4b` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:26:57.826745 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e10d850-349a-43d0-a7bb-2076979f77f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dc2d1a5bde44defae89c15d60172d4b)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:26:57.828673 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e10d850-349a-43d0-a7bb-2076979f77f1 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:26:57.875028 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-1e10d850-349a-43d0-a7bb-2076979f77f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dc2d1a5bde44defae89c15d60172d4b'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:26:57.875643 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-1e10d850-349a-43d0-a7bb-2076979f77f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dc2d1a5bde44defae89c15d60172d4b'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:26:57.887809 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-1e10d850-349a-43d0-a7bb-2076979f77f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dc2d1a5bde44defae89c15d60172d4b'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:26:57.894490 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-1e10d850-349a-43d0-a7bb-2076979f77f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dc2d1a5bde44defae89c15d60172d4b'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:26:57.901888 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-1e10d850-349a-43d0-a7bb-2076979f77f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dc2d1a5bde44defae89c15d60172d4b'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:26:57.907955 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1482/2978] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 15:26:57 2022] DELETE /identity/v3/users/2dc2d1a5bde44defae89c15d60172d4b => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:26:58.080159 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354b345f-a1bf-44d1-b5ae-36b3c0854077 service placement] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:26:58.091914 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354b345f-a1bf-44d1-b5ae-36b3c0854077 service placement] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:26:58.150601 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a613cff-9769-4577-b5d9-17e3bd44547c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:26:58.151157 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a613cff-9769-4577-b5d9-17e3bd44547c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:26:58.151467 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a613cff-9769-4577-b5d9-17e3bd44547c admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:26:58.151742 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a613cff-9769-4577-b5d9-17e3bd44547c admin admin] PATH_INFO: `/v3/projects/b45f3223ea9541f38831d425cb35dbb3` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:26:58.152862 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a613cff-9769-4577-b5d9-17e3bd44547c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b45f3223ea9541f38831d425cb35dbb3)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:26:58.154770 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a613cff-9769-4577-b5d9-17e3bd44547c admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:26:58.201026 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-6a613cff-9769-4577-b5d9-17e3bd44547c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b45f3223ea9541f38831d425cb35dbb3', 'request_id': 'req-6a613cff-9769-4577-b5d9-17e3bd44547c'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:26:58.206549 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-6a613cff-9769-4577-b5d9-17e3bd44547c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b45f3223ea9541f38831d425cb35dbb3', 'request_id': 'req-6a613cff-9769-4577-b5d9-17e3bd44547c'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:26:58.213216 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-6a613cff-9769-4577-b5d9-17e3bd44547c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b45f3223ea9541f38831d425cb35dbb3', 'request_id': 'req-6a613cff-9769-4577-b5d9-17e3bd44547c'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:26:58.227537 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-6a613cff-9769-4577-b5d9-17e3bd44547c admin admin] The token cache is being invalidate because project b45f3223ea9541f38831d425cb35dbb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:26:58.227833 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-6a613cff-9769-4577-b5d9-17e3bd44547c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:26:58.228475 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-6a613cff-9769-4577-b5d9-17e3bd44547c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:26:58.229974 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1497/2979] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 15:26:58 2022] DELETE /identity/v3/projects/b45f3223ea9541f38831d425cb35dbb3 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:26:58.233910 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e10d850-349a-43d0-a7bb-2076979f77f1 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:26:58.235438 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-1e10d850-349a-43d0-a7bb-2076979f77f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:26:58.249763 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e10d850-349a-43d0-a7bb-2076979f77f1 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:26:58.310992 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0437c604-723d-45ca-b17c-969531a74f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:26:58.311704 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437c604-723d-45ca-b17c-969531a74f00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:26:58.311985 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437c604-723d-45ca-b17c-969531a74f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:26:58.312257 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437c604-723d-45ca-b17c-969531a74f00 admin admin] PATH_INFO: `/v3/users/c8f6b756b74242d092c02933c85d7488` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:26:58.313401 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0437c604-723d-45ca-b17c-969531a74f00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8f6b756b74242d092c02933c85d7488)` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:26:58.315342 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0437c604-723d-45ca-b17c-969531a74f00 admin admin] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:26:58.364891 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-0437c604-723d-45ca-b17c-969531a74f00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8f6b756b74242d092c02933c85d7488'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:26:58.365534 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-0437c604-723d-45ca-b17c-969531a74f00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8f6b756b74242d092c02933c85d7488'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:26:58.375724 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:26:58.378179 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-0437c604-723d-45ca-b17c-969531a74f00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8f6b756b74242d092c02933c85d7488'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:26:58.387972 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-0437c604-723d-45ca-b17c-969531a74f00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8f6b756b74242d092c02933c85d7488'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:26:58.394942 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.notifications [None req-0437c604-723d-45ca-b17c-969531a74f00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8f6b756b74242d092c02933c85d7488'} {{(pid=81689) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:26:58.400939 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1483/2980] 10.222.0.30 () {58 vars in 1224 bytes} [Tue Aug 30 15:26:58 2022] DELETE /identity/v3/users/c8f6b756b74242d092c02933c85d7488 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:26:58.575471 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a613cff-9769-4577-b5d9-17e3bd44547c admin admin] Authenticating user token {{(pid=81690) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:26:58.576989 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.fernet_utils [None req-6a613cff-9769-4577-b5d9-17e3bd44547c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81690) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:26:58.592403 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a613cff-9769-4577-b5d9-17e3bd44547c admin admin] Validating token access rules against request {{(pid=81690) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:26:58.602406 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:26:58.653784 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9495bcd7-c7d3-43af-b5d6-a7f0d1402ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a989d2c0a4a49eb9303e4db6cc0b7e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ac908dcbf994029b1c39e157d0de0cb', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81690) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:26:58.654341 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9495bcd7-c7d3-43af-b5d6-a7f0d1402ab6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:26:58.654620 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9495bcd7-c7d3-43af-b5d6-a7f0d1402ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:26:58.654894 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9495bcd7-c7d3-43af-b5d6-a7f0d1402ab6 admin admin] PATH_INFO: `/v3/projects/dfc35a56b0ed4a6c9015e2804059eeb1` {{(pid=81690) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:26:58.656055 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9495bcd7-c7d3-43af-b5d6-a7f0d1402ab6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfc35a56b0ed4a6c9015e2804059eeb1)` {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:26:58.657957 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9495bcd7-c7d3-43af-b5d6-a7f0d1402ab6 admin admin] RBAC: Authorization granted {{(pid=81690) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:26:58.703051 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-9495bcd7-c7d3-43af-b5d6-a7f0d1402ab6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfc35a56b0ed4a6c9015e2804059eeb1', 'request_id': 'req-9495bcd7-c7d3-43af-b5d6-a7f0d1402ab6'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:26:58.708624 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-9495bcd7-c7d3-43af-b5d6-a7f0d1402ab6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfc35a56b0ed4a6c9015e2804059eeb1', 'request_id': 'req-9495bcd7-c7d3-43af-b5d6-a7f0d1402ab6'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:26:58.714546 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-9495bcd7-c7d3-43af-b5d6-a7f0d1402ab6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfc35a56b0ed4a6c9015e2804059eeb1', 'request_id': 'req-9495bcd7-c7d3-43af-b5d6-a7f0d1402ab6'} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:26:58.727188 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-9495bcd7-c7d3-43af-b5d6-a7f0d1402ab6 admin admin] The token cache is being invalidate because project dfc35a56b0ed4a6c9015e2804059eeb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81690) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 15:26:58.727502 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-9495bcd7-c7d3-43af-b5d6-a7f0d1402ab6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:26:58.728116 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG keystone.notifications [None req-9495bcd7-c7d3-43af-b5d6-a7f0d1402ab6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81690) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 15:26:58.729571 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: [pid: 81690|app: 0|req: 1498/2981] 10.222.0.30 () {58 vars in 1233 bytes} [Tue Aug 30 15:26:58 2022] DELETE /identity/v3/projects/dfc35a56b0ed4a6c9015e2804059eeb1 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 15:27:08.397432 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:DELETE=2,keystone:SELECT=7,keystone:INSERT=1 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:27:08.723407 dv-cinder-828806-5-iscsi devstack@keystone.service[81690]: DEBUG dbcounter [-] [81690] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81690) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 15:31:12.534151 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0437c604-723d-45ca-b17c-969531a74f00 admin admin] Authenticating user token {{(pid=81689) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 15:31:12.535627 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-0437c604-723d-45ca-b17c-969531a74f00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:31:12.639639 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0437c604-723d-45ca-b17c-969531a74f00 admin admin] Validating token access rules against request {{(pid=81689) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 15:31:12.698485 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4905bc71-930d-47a5-a547-9b86865ce9e5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcf302a170934c25b39699088dac326a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01cf71ca5abf4576b89d10c85af6d2de', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81689) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 15:31:12.699116 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905bc71-930d-47a5-a547-9b86865ce9e5 service placement] REQUEST_METHOD: `GET` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 15:31:12.699493 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905bc71-930d-47a5-a547-9b86865ce9e5 service placement] SCRIPT_NAME: `/identity` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 15:31:12.699768 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905bc71-930d-47a5-a547-9b86865ce9e5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81689) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 15:31:12.701504 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.fernet_utils [None req-4905bc71-930d-47a5-a547-9b86865ce9e5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81689) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 15:31:12.714140 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4905bc71-930d-47a5-a547-9b86865ce9e5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 15:31:12.715740 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4905bc71-930d-47a5-a547-9b86865ce9e5 service placement] RBAC: Authorization granted {{(pid=81689) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 15:31:12.725415 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: [pid: 81689|app: 0|req: 1484/2982] 10.222.0.30 () {60 vars in 1365 bytes} [Tue Aug 30 15:31:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 15:31:22.721234 dv-cinder-828806-5-iscsi devstack@keystone.service[81689]: DEBUG dbcounter [-] [81689] Writing DB stats keystone:SELECT=71 {{(pid=81689) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}